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14.  ABSTRACT  Background:  Advances  in  health  information  technology  (HIT)  and  the  use  of  evidence-based  (EB),  clinical 
decision  support  (CDS)  tools  in  electronic  health  records  (EHR)  hold  great  promise.  Researchers  report  that  EHR  technology 
may  be  an  effective  vehicle  for  providing  EB  information  to  clinicians,  but  little  is  known  about  how  electronic  innovations 
work  to  support  nurses  to  know  and  use  best  practices  to  achieve  optimal  patient  outcomes. 

Objective:  This  study  was  designed  to  evaluate  the  impact  of  the  Knowledge -Based  Nursing  (KBN)  innovation,  a  customized 
design  featuring  actionable  EB  recommendations  embedded  into  policy  and  the  content  and  CDS  tools  in  the  EHR  to  support 
nurses  to  use  best  practices  for  six  phenomena  (pain,  medication  adherence,  depression/suicide,  fall  risk,  pressure  ulcer 
risk/actual,  and  delirium)  to  improve  patient  outcomes.  The  study  was  guided  by  the  Dissemination  and  Implementation  of 
Evidence-based  Policy  Framework  (adapted  from  Dodson,  Brownson,  &  Weiss,  2012)  to  evaluate  the  impact  of  the  innovation 
under  usual  deployment  conditions  and  to  see  if  unit -based  implementation  strategies  could  improve  the  effectiveness  of  the 
innovation  over  time. 

Hypothesis  1:  The  KBN  Innovation,  deployed  with  passive  dissemination,  will  have  a  positive  effect  on  nursing  knowledge, 
use  of  evidence-based  practices,  and  the  achievement  of  nurse-sensitive  patient  outcomes  at  baseline. 

Hypothesis  2:  “Active”  implementation  (audit/feedback  of  baseline  results,  education  with  behavioral  expectations,  leader- 
driven  unit  implementation  and  maintenance)  by  nurse  leaders  and  designated  staff  will  improve  nurse  knowledge  and  use  of 
electronic  and  clinical  practices  and  produce  measurable  improvements  in  outcomes  compared  to  passive  dissemination  alone. 
Methods:  This  pre/post  mixed  methods  study  was  conducted  with  consenting  medical/surgical  and  critical  care  nursing  units 
(N=23)  of  a  quaternary  medical  center  where  the  KBN  Innovation  was  deployed.  The  KBN  Logic  Model  guided  the 
development  of  nurse  and  patient  surveys,  audits/reports,  and  non-participant  observation  methods  used  to  gather  detailed 
information  about  unit  context,  nurse  characteristics,  nurse  knowledge,  care  processes,  patient  perceptions,  and  outcomes  for 
each  unit.  A  multimodal  implementation  intervention  was  created  to  address  knowledge  and  behavior  gaps  identified  at 
baseline.  The  intervention  included  audit-based  feedback  and  training  (3.5  hrs)  for  staff  and  leaders  to  address  knowledge 
gaps.  Units  identified  their  priorities  and  worked  on  unit -based  implementation  with  varied  maintenance  strategies  during  the 
subsequent  6  month  interval. 

Results:  The  study  yielded  much  data  about  staff  nurses  and  nurse  leaders  and  how  they  worked  to  deliver  EB  care  to  patients 
in  the  real  world.  At  baseline  (Q2,  2014),  the  electronic  KBN  CDS  tools  functioned  as  designed.  The  observations  and  audits 
revealed  that  staff  utilized  the  KBN-based  technology  to  support  them  to  assess,  diagnose,  and  deliver  EB  care.  Gaps  were 
identified  in  nurse/nurse  leader  knowledge  scores  (M=55.3%  correct,  SD  8.5%).  Baseline  leader  observations  revealed  limited 
leader  oversight  of  staff  practice.  Patients  reported  positive  educational  interactions  with  evidence  of  knowledge  gain.  The 
optimization  feedback  and  training  was  delivered  to  leaders  (N=48-100%)  and  staff  nurses  (849,  90%)  with  subsequent  unit 
implementation.  Units  reviewed  their  baseline  results  and  identified  their  outcome  improvement  priorities.  Staff  preceptors 
received  grant-supported  time  to  assist  leaders  to  implement  and  maintain  best  practices  for  each  unit  (up  to  64  hrs/unit)  but 
they  utilized  usual  practices  (without  the  use  of  reports).  In-depth  analysis  of  the  observations  and  audits  are  still  in  progress. 
Conclusion:  The  conceptual  framework  was  useful  in  guiding  the  analysis.  Gaps  in  the  knowledge  and  use  of  EBP  behaviors 
were  addressed  by  the  implementation  intervention  with  a  high  percentage  of  participation.  Unit  implementation  was  carried 
out  using  usual  practices  (without  the  use  of  reports  and  staff  follow-up).  Preliminary  findings  indicate  that  there  was  little 
improvement  in  the  outcomes.  Additional  analysis  remains  in  progress  to  identify  details  that  may  explain  these  findings. 

NOTE:  A  Federal  Military  Advisory  Committee  is  involved  in  all  phases  of  the  project  to  review  and  provide  input/feedback 
on  the  study  and  identify  ways  to  disseminate  findings  and  applications  for  the  military. 
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INTRODUCTION 

This  pre/post  mixed  methods  research  study  is  designed  to  evaluate  the  impact  of  Knowledge- 
based  Nursing  (KBN)  -  a  technological  innovation  that  features  customized  evidence-based 
content  and  clinical  decision-support  (CDS)  tools  in  the  electronic  health  record  (EHR) 
specifically  designed  to  support  nurses  to  know  and  use  evidence-based  practices  (EBP)  in  their 
patient  care.  The  project  is  on  track  with  the  projected  milestones  and  timeline  for  the  primary 
study  sight  with  expenses  under  the  established  budget. 

This  report  provides  a  summary  of  our  achievements  over  the  past  three  years.  We  completed 
our  baseline  assessment  (Q2,  2014)  and  identified  that  the  technology  functioned  as  designed  but 
there  were  staff  and  leader  knowledge  and  practice  gaps  that  were  believed  to  have  an  impact  on 
the  effectiveness  of  the  tool.  We  collaborated  with  international  experts  in  the  field  of 
dissemination  and  implementation  (D  &  I)  science  to  utilize  a  D  &  I  conceptual  framework  to 
design  a  multimodal  implementation  intervention.  The  experts  advised  us  to  evaluation  varied 
implementation  strategies  (A  vs.  B)  since  it  is  already  known  that  dissemination  -  alone  has  little 
impact.  We  summarized  the  audit -based  feedback  and  used  it  to  create  the  training  curriculum 
and  materials  aimed  at  the  identified  deficits.  The  training  sessions  (n=44)  were  attended  by 
100%  of  the  nurse  leaders  and  90%  of  staff  nurses  (Sept/Oct,  2014).  We  provided  units  with 
audit -based  feedback  results  and  supported  them  to  implement  and  use  varied  strategies  to 
monitor/maintain  the  EBP  over  the  months  that  followed.  Post-intervention  observations  and 
survey  assessments  were  completed  during  Q2,  2015.  The  Team  continues  to  work  on 
processing  and  thoroughly  analyzing  the  findings  from  our  primary  site. 

NOTE:  The  Team  received  approval  to  repurpose  unused  funds  from  Years  1  and  2  to  replicate 
the  study  at  two  small  non-urban,  non-Magnet,  community  hospitals  to  enhance  study 
generalizability.  A  no-cost  9  month  extension  was  approved  to  support  study  replication  and 
subsequent  analysis. 

This  study  represents  an  in-depth  evaluation  of  an  innovative  strategy  used  to  support  nurses  and 
nurse  leaders  to  know  and  consistently  use  evidence-based  practices  in  acute  care.  We  gathered 
an  extensive  amount  of  infonnation  about  the  unit  context,  staff  and  nurse  leader  characteristics, 
nurse  knowledge  ,  observations  and  audits  to  evaluate  the  use  and  documentation  of  evidence- 
based  practices  and  the  impact  on  nursing  sensitive  outcome  at  a  large  urban  Magnet-designated 
medical  center  where  it  was  deployed.  We  are  preparing  several  manuscripts  detailing  the  study 
protocol,  the  theory,  the  methodology,  and  eventually  the  findings. 

KEYWORDS  [MESH] 

•  Decision  Support  Systems,  Clinical 

•  Dissemination,  Information. 

•  Evidence-Based  Nursing 

•  Evidence-based  Practice 

•  Health  Services  Research 

•  Nursing  Evaluation  Research 

•  Nursing  Informatics 

•  Nursing  Process  Patient  Care 

•  Organizational  Context 
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•  Outcome  and  Process  Assessment  (Healthcare) 

•  Patient  Care  Planning 

•  Perfonnance  metrics 

•  Research  methodology 


OVERALL  PROJECT  SUMMARY 

Year  1  Summary:  The  KBN  Impact  Study  Research  Team  started  our  work  by  reviewing  the 
KBNI  conceptual  framework  and  Logic  Model  (Appendix  A),  and  the  KBN  evidence  summaries 
to  identify  essential  components.  We  reviewed  the  literature  to  identify  a  suitable  way  to 
measure  context  and  factors  that  influence  the  knowledge  and  use  of  evidence-based  practices  in 
patient  care  with  appropriate  measures  (or  create  some  if  not  available).  The  team  selected  an 
established  tool,  the  Alberta  Context  Tool  (Estabrook,  Squires,  Cummings,  Birdsell,  Norton, 
2009)  and  the  associated  demographic  and  research  utilization  questions  to  support  measurement 
and  comparability.  The  team  developed  tools  to  assess  nurse  and  nurse  leader  knowledge  about 
the  essential  practices  related  to  the  six  phenomena  in  focus  for  the  study.  We  also  created  tools 
to  assess  the  use  of  essential  practices  during  patient  care  with  non-participant  observations  and 
audits  of  the  electronic  record  where  the  nurses  documented  the  care  they  provided.  Tools  were 
also  developed  to  audit  documented  patient  education  and  to  subsequently  complete  a  guided 
interview  (“patient  survey”)  to  gather  patient  perceptions  and  results  of  evidence-based  patient 
education  that  they  had  received.  We  also  collected  descriptive  information  about  the  units  and 
the  nurse  sensitive  outcomes  that  were  reported  for  each  unit  for  the  study  period.  The  baseline 
findings  were  designed  to  evaluate  the  use  of  the  technology  and  the  essential  practices  under 
usual  deployment  conditions  that  were  in  place  at  the  study  institution.  Usual  deployment 
conditions  consist  of  hands-on  EHR  functionality  training  (3-8  hour  days)  for  go-live  or  with 
orientation  if  nurses  were  hired  after  go-live.  Units  have  their  own  process  for  disseminating 
policies  and  KBN-based  on-line  training  modules  to  their  staff.  The  Research  Team  also 
validated  that  the  essential  content  and  CDS  functionalities  were  working  as  designed  in  the 
EHR.  The  Team  requested  two  enhancements  to  allow  nurses  to  view  risk  factors  and  learning 
assessments  during  their  care.  The  team  refined  the  protocol  for  data  collection  and  tested  the 
tools  and  procedures  in  an  alternate  site  prior  to  use  for  the  study.  The  tools  and  study  protocol 
were  pre-reviewed  by  the  TATRC  Regulatory  Compliance  Specialist  and  received  expedited 
review  and  approval  by  the  Aurora  Health  Care  (AHC)  Institutional  Review  Board  (IRB)  (20- 
DEC-2013)  and  the  Human  Research  Protection  Office  (HRPO)  for  the  US  Anny  Medical 
Research  and  Material  Command  (21-FEB-2014).  In  addition  to  the  research  study,  the  KBN 
Team  collaborated  with  the  TATRC  Contracting  Officer  Representative  -  Ollie  Gray  to  establish 
the  Federal  Military  Advisory  Group.  The  Advisory  Group  was  led  by  LTC  Michael  Ludwig, 
RN-BC,  MS,  CPHIMS,  AMEDD  Chief  Nursing  Infonnation  Officer  and  included  members  of 
the  Federal  Nursing  Informatics  iEHR  Collaborative.  The  group  developed  a  charter  and  held 
ongoing  meetings  at  key  stages  of  the  project. 

Year  2  Summary:  After  receiving  Aurora  Institutional  Review  Board  (IRB)  and  HRPO 
approvals,  the  KBN  Research  Team  immediately  informed  and  recruited  nursing  units  (N=23) 
into  the  study.  A  power  analysis  indicated  that  group  sample  sizes  of  337  observations  would  be 
needed  for  two  sample  t-test  evaluation  to  achieve  90%  power  to  detect  a  small  difference  (0.30) 
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with  alpha  set  at  0.05.  The  team  created  a  4  month  study  calendar  to  schedule  observation 
sessions  (4-6  hour  sessions  for  medical-surgical  units  and  9-6  hour  sessions  for  ICU  units)  to 
observe  approximately  16-19  patients  per  unit  for  optimal  power.  Observation  and  patient 
survey  days  were  scheduled  to  occur  when  students  were  not  scheduled  to  have  clinical  on  the 
unit.  The  Patient  Survey  data  collection  calendar  for  medical/surgical  units  was  prepared  with  2 
days  per  unit  to  enroll  5%  (approximately  10  patients)  per  unit).  The  calendar  was  arranged  to 
focus  on  a  set  group  of  units  each  month,  collect  all  observations  and  patient  interview  within 
the  same  month.  The  baseline  data  were  collected  and  analyzed.  The  findings  identified  that 
significant  gaps  in  knowledge  were  present  across  all  6  of  the  phenomena.  Leaders  scored 
significantly  higher  than  staff,  but  there  were  no  differences  in  scores  by  unit  or  unit  type.  The 
Team  used  the  essential  practice  summaries  and  baseline  findings  to  create  a  list  of  behavioral 
expectations  and  the  plan  for  delivering  the  implementation  intervention.  The  Team  prepared  the 
curriculum  and  materials,  completed  continuing  education  credit  paperwork,  and  set  up  the 
speaker/participant  schedule  and  procedures  for  recruiting  and  delivering  44-3.5  hour  training 
session  to  all  nursing  leaders  (N=45)  and  staff  (N=  849).  In  addition  to  knowledge-deficits,  the 
observations  revealed  that  nurse  leader  oversight  and  support  for  the  use  of  the  essential  practices 
was  limited  -  observed  primarily  during  daily  team  meetings.  The  intervention  was  adjusted  to 
provide  funding  to  allow  (2)  staff  nurses  who  were  involved  in  orienting  new  staff  (preceptors) 
to  have  grant-supported  time  to  support  implementation  on  the  unit. 

The  PI  increased  her  knowledge  of  dissemination  and  implementation  (D&I)  science  by 
participating  in  the  NIH  Training  D&I  Training  Institutes.  The  D&I  experts  advised  the  PI  to 
be  more  explicit  in  describing  the  core  components  of  the  KBN  innovation  (Appendix  B)  and  the 
importance  of  multimodal  implementation  strategies.  Based  on  expert  input,  the  PI  and  the  team 
reviewed  the  “Dissemination  and  Implementation  of  Evidence-based  Policy”  framework 
(Dodson,  Brownson,  and  Weiss,  2012)  and  adapted  it  (Appendix  C)  to  capture  the  essential 
concepts  important  to  the  current  study  with  the  addition  of  dissemination  and  implementation 
process  descriptions  and  intermediary  outcomes  (clinician  knowledge  and  use  of  EBP)  that 
appear  to  be  an  essential  step  toward  achieving  optimal  patient  outcomes. 

Year  3  Summary:  The  Team  provided  informal  follow  up  to  support  unit  leaders  and  preceptors 
to  focus  on  their  priorities  and  to  use  usual  processes  (Strategy  A)  or  electronic  reports  (Strategy 
B)  to  monitor  and  maintain  adherence  to  the  essential  practices.  The  post-intervention 
assessment  was  carried  out  based  on  the  protocol  during  Q2  2015.  The  remainder  of  the  year  has 
been  spent  downloading,  cleaning,  and  analyzing  the  findings  at  the  primary  site  using  the 
conceptual  frameworks  to  guide  the  interpretation. 

The  Team  received  approval  for  their  proposal  to  repurpose  unused  funds  to  replicate  the  study 
at  two  community  hospitals,  sampling  inpatient  nursing  units  with  diverse  characteristics  from 
the  primary  study  site.  Four  additional  units  were  recruited  to  the  study  (N=90  RN).  Baseline 
data  collection  was  completed  in  September  with  nonparticipant  unit  (N=5 1)  and  admission 
(N=10)  observations,  patient  interviews  (N=27)  and  nurse  survey  participants  (N=50,  55%). 

Personnel  and  Project  Administration 
Years  1-3: 

•  AHC  completed  a  legal  review  and  created  a  financial  cost  center  for  the  project  (FEB-2013) 
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•  The  Project  Kick  Off  Meeting  was  held  in  MAR-  2013  with  on-boarding  of  Research  Team 
include  Researchers  (Hook,  Gentile,  &  Singh)  and  Nursing  Infonnatics  Staff  (obtained  new 
research  credentials  and  CITI  Training);  Nursing  Infonnatics  Specialist  positions  and  hours 
were  adjusted  so  staff  were  available  to  complete  the  nonparticipant  observations  efficiently. 

•  Recruited,  hired  and  on-boarded  two  Clinical  Nurse  Specialist  (CNS)  advisors  to  support 
communication  and  subject  recruitment  -  NOV  2013 

•  Recruited  and  on-boarded  a  nurse  leader  (site  Chief  Nurse  Executive)  to  serve  as  the  Patient 
Care  Manager  advisor  to  support  Manager  recruitment  and  participation  (participates  with 
no  reimbursement  since  salaried  with  limited  hours)-  JAN-2014 

•  Job  descriptions  were  created  &  posted  for  Project  Manager  (PM)  &  Graduate  Intern;  An 
interim  PM  (part  time)  helped  to  set  up  the  project  plan  (PM  vacant  for  5  months) 

•  Interviewed,  hired,  and  on-boarded  Project  Manager  -  JUN-2013  through  JAN-2014; 

Position  was  vacant  for  4  months  until  a  replacement  was  hired  (MAY-2014) 

•  Interviewed,  hired,  and  on-boarded  Graduate  Intern  -  JUN-  2013;  Intern  was  hired  into  a 
Researcher  position  when  she  finished  her  coursework  DEC-2013  through  AUG-2014. 
Interviewed,  hired,  and  on-boarded  a  replacement  research  scientist  (JUL-2014)  and  added 
another  (OCT-2014)  for  observation  data  entry,  chart  audit/entry,  data  cleaning,  and  analysis. 

•  The  study  site  “Magnet”  Program  Manager  was  added  as  a  clinical  advisor  to  support  unit 
implementation  and  preceptor  skills  data  collection  -  FEB-2015 

•  Submitted  a  revised  Statement  of  Work  (SOW)  and  Letter  to  the  Contracting  Officer 
Representative  (COR)  Tony  Story,  requesting  to  repurpose  funds  to  replicate  the  study  at  two 
community  hospitals  to  increase  the  sample  diversity  and  broaden  generalizability. 

(Approved  JUL-2015). 

•  Research  Scientist,  (Badger)  transferred  AUG-2015;  Research  Scientist  (Bauer)  rehired  28- 
SEPT-2015. 

•  Kicked  off  the  KBN  Impact  Study  at  two  additional  community  hospitals  and  replicating 
KBN  Impact  Study  Goals  4,  6,  7  &  8  with  new  sites  -  see  details  below  (JULY-2015) 

•  In  Progress  Review  (IRP)  presented  at  Fort  Detrick,  MD  (DEC-2015) 

Consultants: 

•  The  AHC  Research  Administration  Contract  Office  set  up  fonnal  contracts  with  two 
consultants  (Devine  and  Dowding)  named  in  the  proposal 

•  The  Study  Team  collaborated  with  Dr.  Beth  Devine  to  complete  the  essential  knowledge 
review  and  begin  to  draft  the  knowledge  test  questions  for  nurse/nurse  leaders  (Consultant 
work  completed  with  4  additional  hours  of  effort  above  budget)  -  Contract  ended  JAN-2014 

•  The  Study  Team  collaborated  with  Dr.  Dawn  Dowding  regarding  study  design.  During  Year 

1,  Dr.  Dowding  transitioned  her  faculty  position  from  the  University  of  Leeds  (UK)  to 
Columbia  University  (USA).  Her  initial  contract  was  finalized  for  effort/consultation  in  Year 

2.  She  reviewed  the  study  protocol  and  measures  and  provided  feedback.  She  conducted  a 
site  visit  in  MAY-2014.  She  received  an  orientation  to  the  KBN  Innovation  and  observed  all 
facets  of  the  baseline  data  collection  that  was  in  progress.  She  provided  valuable  feedback  to 
the  study  team. 

•  The  KBN  Impact  Study  has  many  opportunities  to  share  the  work  and  findings  from  this 
study.  To  address  this  need,  the  PI  recruited  a  scientific  writing  consultant  (Chris 
McLaughlin)  -  APR-2014  to  support  the  team  to  create  a  dissemination  plan  and  to  develop  a 
team  approach  for  abstract,  poster,  and  manuscript  preparation  during  Years  2  and  3. 
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•  Researchers  collaborated  with  the  consultants  periodically  throughout  2015.  Contracts  were 
extended  to  utilize  the  remaining  hours  in  2016 

GOAL  #1:  Identify  essential  knowledge  and  nursing  practice  behaviors  (components) 

STATUS:  Completed  Milestone  in  collaboration  with  Consultant  -  29-JUN-2013 

•  The  KBN  Research  Team  reviewed  and  systematically  analyzed  the  evidence-based  practice 
synthesis  documents  to  identify  essential  knowledge  &  practice  behaviors  for  six 
phenomena:  Acute  Pain,  Medication  Non-adherence,  Depressive  Symptoms/ Suicide,  Risk 
for  Falls/Fall-related  Injury/Post  Fall  Management,  Pressure  Ulcer  Risk/Actual,  and  Delirium 
Risk/Actual-all  Venues  (ICU  and  Med-Surgical) 

•  The  KBN  Research  Team  conducted  iterative  process  meetings  to  identify  the  “essential” 
components  -  defined  as  those  knowledge  or  behavior  components  that  are  necessary, 
indispensable,  and  foundational  for  staff  and/or  nurse  leaders  to  carry  out  the  patient  care  or 
meet  the  expected  outcome/goal.  A  spreadsheet  was  create  to  support  the  analysis: 

o  Recommendations  from  the  synthesis  regarding  the  assessments,  diagnoses, 
interventions,  and  outcomes  for  each  phenomena 
o  Population  specific  requirements  based  on  age  or  risk  factors 
o  Details  about  how  component  is  entered  into  the  EHR/functionality  (e.g.  content  or 
clinical  decision) 

o  Details  about  where  the  component  is  documented  (e.g.  flow  sheet/Patient 
Education/Care  Plan/Medication  Administration  Record,  etc.) 
o  Details  about  how  the  researcher  knows  the  component  was  completed 
o  Details  about  how  the  researcher  know  if  a  CDS  tool  was  used  correctly 
o  Details  about  which  components  are  embedded  into  a  policy  or  standard 

GOAL  #2:  Validate  that  essential  KBN  electronic  content/tools  are  incorporated  in  the 

electronic  health  record  (EHR)  and  functioning  as  designed 

STATUS:  Completed  Milestone  w/addl  build  to  support  manual  screening  -  12-DEC-2013 

•  Utilized  findings  from  Goal  #1  as  the  basis  for  the  gap  identification  conducted 
simultaneously  during  syntheses  review  of  essential  knowledge  and  nursing  practice 
behaviors  (preliminary  list  of  gaps  identified). 

•  Submitted  specifications  (17-Jun-2013)  for  building  the  “sidebar  report”  a  print  group  report 
that  provides  nurses  with  viewable  infonnation  about  patient  risk  factors  for  use  in  matching 
interventions  and  patient  education.  Completed  and  tested. 31  -Aug-20 13 

•  Submitted  specifications  for  building  manual  mechanism  for  initiating  additional  screening 
tools  even  if  they  do  not  trigger  based  on  patient  assessment  on  admission  -  DEC  2013 

•  Submitted  specifications  for  daily  and  monthly  electronic  report  for  capturing  depressive 
symptom,  cognitive  and  medication  adherence  screening  on  the  Key  Performance  Indicator 
daily  and  Monthly  reports.  NOV-DEC  2013. 


GOAL  #3:  Develop  reliable  and  valid  measures  and  measurement  processes  for  evaluating 
the  implementation  and  adoption  of  KBN-based  practices 
STATUS:  Completed  Milestone  -  DEC-2013; 

Additional  metrics  were  added  (Modification  #9)  15-FEB-2015 
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Measure  Development  (refer  to  details  in  Research  Accomplishments) 

•  Nurse  Survey  -  4  part  tool  including  the  Alberta  Context  Tool,  demographics,  research 
utilization  questions,  and  KBN  Knowledge  Test  (41  questions  evidence-based  practices 
described  in  system  nursing  policy  based  on  KBN) 

•  Nonparticipant  Observation  Tools  during  admission  and  ongoing  patient  care 

•  Audit  Tool  for  evaluating  associated  documentation  during  admission  and  ongoing  care 

•  Patient  Survey  including  preliminary  medical  record  review 

•  Process  and  Outcome  Metrics:  Process  and  outcome  metrics  were  identified  in  the  study 
protocol  and  will  extracted  from  existing  sources. 

•  Preceptor  background  characteristics  were  collected  with  time  and  activity  tracking  form 
(Appendix  E). 

•  Preceptor  knowledge  and  ability  to  use  the  monitoring  tools  appropriate  to  their  study  group 
(A  -  usual  care  vs.  B  -  KPI  Daily  and  other  electronic  reports)  was  evaluated  during  the 
month  prior  to  post-implementation  assessment.  (Appendix  E). 

GOAL  #4:  Conduct  baseline  measurement  to  identify  gaps  (knowledge,  practice  behaviors, 

or  EHR  build)  to  improve  the  integrity  of  the  planned  KBN  intervention  study 

STATUS:  Completed  Milestone  -  30-JUN-2014 

•  AHC  Biomedical  Institutional  Review  Board  (IRB)  Study  #13-142E  approved  the  study  with 
expedited  review  a  with  waiver  of  documentation  of  infonned  consent  for  nurse  subjects, 
HIPAA  authorization  for  retrospective  medical  record  review,  and  requirement  for 
maintaining  a  copy  of  the  patient  subject  consent  in  the  subject’s  medical  record. 

•  DOD/USAMRMC  Award  #W81XWH-13-l-0034  protocol  was  submitted  for  review  to  the 
US  Department  of  Defense  Human  Research  Protection  Office  (HRPO)  JAN-  2014 

•  Brigit  Ciccarello,  M.A.,  Regulatory  Compliance  Specialist,  Telemedicine  &  Advanced 
Technology  Research  Center  (TATRC)  Research  Program  Officer  advised  proceeding  with 
the  administrative  steps  for  unit/subject  recruitment  with  initiation  of  data  collection  once 
HPRO  approval  was  received. 

•  Recruitment  meetings  were  kicked  off  with  study  site  Nurse  Leaders  on  7- Jan-20 13.  A 
recruitment  video  was  created  to  support  a  consistent  message  to  all  eligible  units/nurses. 
Unit-level  recruitment  meetings  were  held  with  the  use  of  a  recruitment  video.  Unit 
recruitment  was  completed  28-FEB-2014  with  all  units  (N=23)  agreeing  to  participate. 

•  Baseline  assessments  were  carried  out  between  1  l-MAR  through  30-JUN-2014 

o  Nonparticipant  Observations  (N=379  RN/Patient  observations,  54  Nurse  Leaders 
observations,  and  40  RN/Patient  admission  observations)-  Completed  27-May-2014 
Observations  were  conducted  per  protocol  on  all  the  study  units  (6  hour  sessions) 
Med/Surgical  Units  =  4  observations/unit  (approximately  25  patients/unit) 

Critical  Care  Units  =  9  observations  each  (approximately  20  patients/unit) 

Admissions  (n=2/unit)  were  observed  (3  units  did  not  complete  admission  observations 
because  patient  admission  processes  were  completed  by  another  unit). 

Nonparticipant  Observations  were  gathered  using  (2)  paper-based  tools  and  transcribed 
into  an  electronic  data  collection  tool  based  in  SurveyMonkey  ™  software. 

Data  entry  for  nurse  leader  observations  was  completed  15-AUG-2014  and  for  non¬ 
participant  observations  by  SEPT-2014 
o  Audits  (N=379  +  40  Admissions) 
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Near  real  time  auditing  was  not  possible  because  most  staff  worked  12  hour  shifts  with 
data  entry  extending  beyond  the  end  of  the  observation  time.  The  audits  were  conducted 
retrospectively  using  the  established  process.  Audits  took  much  longer  than  to  complete 
(approximately  45  minutes/observation)  than  initial  estimate.  Data  entry  for  baseline 
audits  was  completed  JAN-2015 

o  Patient  Survey-  Study  Period  MAR-2014  through  MAY-2014 

Patients  (n=185)  were  recruited  and  interviewed  per  protocol  on  the  18  non-ICU  units 
during  the  study  period.  Chart  audits  and  interview  data  were  entered  using  an  iPad  into 
Survey  Monkey.  Data  were  downloaded  into  excel,  cleaned,  and  analyzed.. 

o  Nurse  Survey  -  Study  Period  27-May-20 14  through  30-JUN-2014 

The  Nurse  Survey  Tool  was  opened  for  data  collection  after  unit  observation  were 
completed  to  minimize  staff  awareness  of  the  practices  that  were  being  tested  and 
observed  for  during  the  nonparticipant  observations.  The  Nurse  Survey  was  “kicked”  off 
with  the  Nurse  Leaders  with  fliers  and  email  message  sent  to  staff  nurses  employed  on 
the  study  units  and  hospital  float  pool.  The  link  to  the  Nurse  Survey  was  delivered  to 
eligible  participants  using  the  Learning  Connection.  The  research  team  monitored 
participation  and  provided  participation  reports  to  the  unit  nurse  leaders  to  support 
recruitment. 

o  Process  and  Outcome  Metrics :  Unit-based  nurse  sensitive  outcome  data  were  gathered 
from  various  sources  including  the  EHR-based  electronic  reports  (e.g.  KPI  Monthly 
report  re:  compliance  with  standards  and  documentation),  hospital  census  (e.g. 
Patients/Patient  Days,  Length  of  Stay),  and  the  National  Database  for  Nursing  Sensitive 
Indicators  reports  (e.g.  Total/RN  Hours  per  Patient  Day,  Falls/Injuries,  Pressure  Ulcers) 
and  patient  satisfaction  (Hospital  Consumer  Assessment  of  Healthcare  Providers  and 
Systems/HCAPS)  reports  reported  to  external  monitoring  company  used  by  the  study 
institution.  All  units  received  a  summary  outcome  report  of  measures  pertinent  to  the 
study  as  part  of  the  audit/feedback  component  of  the  intervention  to  support  nurse  leaders 
to  identify  priorities  for  implementation  on  the  unit  (Refer  Appendix  D  -  Baseline 
Findings  for  a  sample).  Further  analysis  of  outcomes  is  in  progress. 


GOAL  #5  Design  the  Intervention  Study  strategy  including  the  delivery  method 

STATUS:  Completed  milestone  -  30- AUG-20 14 

•  Developed  schedule  and  logistics  for  Optimization  Training  sessions  to  accommodate  all  942 
eligible  staff  nurses  using  electronic  registration  tracking  system  (Learning  Connection).  30- 
JUNE-2014  (Note:  The  dates  and  rooms  had  to  be  set  up  in  advance  to  allow  nurse  leaders 
to  preplan  classes  and  unit  staffing  and  ensure  room  availability.) 

•  Findings  from  the  Patient  and  Nurse  Survey  findings  and  Nurse  Leaders  Observations  were 
analyzed  and  used  to  identify  knowledge  gaps  -  completed  15-AUG-2014 

•  Findings  from  the  Nurse  Leader  Observations  were  analyzed  used  to  identify  behavior  gaps. 
Given  that  nurse  leaders  had  limited  time  and  focus  on  Additional  resources  to  support 
implementation,  e.g.,  additional  funding,  preceptors,  training,  etc. 

•  Nurse  Leaders  were  asked  to  identify  priority  outcomes  and  essential  behaviors  that  they 
wanted  to  implement  and  maintain  on  their  unit.  Leaders  were  given  (voluntary)  access  to 
additional  study  funding  to  provide  indirect  paid  time  for  (2)  staff  nurse  preceptors  (infonnal 
opinion  leaders)  to  monitor/maintain  the  use  of  best  practices  as  directed  by  the  leaders. 
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•  PI  participated  in  the  2014  Training  Institute  for  Dissemination  and  Implementation  Research 
in  Health  (TIDIRH)  and  networked  with  national/intemational  experts  to  adapt  the 
conceptual  framework  (Appendix  A)  and  to  confirm  the  best  study  intervention 
(audit/feedback,  training  with  behavioral  objectives,  unit  implementation,  and  maintenance) 
with  a  plan  to  randomize  based  on  strategy  for  monitoring  implementation.  25-JUL-2014 

•  Units  were  randomized  into  two  groups  (A  &  B)  based  on  difference  in  strategy  used  to 
monitor  implementation  (usual  care  vs.  electronic  monitoring  using  reports)  (Appendix  E) 

•  Finalized  fonnat,  learning  objectives,  methods,  behavioral  expectations,  training  materials 
and  evaluation  for  the  Nurse  Leader  and  Staff  Nurse  Optimization  Training  course  featuring 
an  overview  with  four  “break-out”  sessions  focusing  on  identified  knowledge  gaps  including: 

o  Navigator/Flowsheets/Care  Planning 
o  Mental  Status  (Delirium  Risk/Actual)/Depressive  Symptoms 
o  Pain  -  Comfort/Function 
o  Patient  Education/Medication  Nonadherence 

•  Developed  (2)  training  videos  to  deliver  study  overview  and  audit/feedback  results  at 
baseline)  and  (6)  brief  videos  to  demonstrate  key  training  content  (e.g.  mental  status 
assessment  (4),  ADL  assessment  (1),  and  depression  screening)  31-AUG-2014 

•  Created  training  materials  for  the  training  sessions: 

o  Hand-outs:  The  8-page  handout  included  an  overview  describing  the  KBN  core 
components  and  a  list  of  the  “essential  practices”  for  implementation  on  the  unit  and 
worksheets  for  each  session  to  practice  the  documentation  during  the  case  studies. 
(Duplicated  and  collated  950  sets)  (Appendix  E) 
o  Reference  Materials:  25  folders  containing  15  printed  reference  sheets  of  content 
available  in  the  EHR  for  participants  to  reference  throughout  the  sessions, 
o  Humorous  incentive:  “BINGO”  game  with  template  filled  with  key  KBN  words  to 
enhance  participant  interaction.  Winners  received  “I  Won  at  KBN  BINGO”  button 
and  were  encouraged  to  wear  them  on  their  units  to  promote  the  training  &  encourage 
adoption. 

•  Completed  the  continuing  education  credit  application  including  speaker  biography  and 
conflict  of  interest  review  and  support/budget  letter  from  USAMC  sponsor.  The  course  was 
awarded  3.67  contact  hours  from  the  Wisconsin  Nurses  Association. 

•  Developed  a  “Trainer  Schedule”  for  KBN  team  and  worked  through  Outlook  to  block 
schedules  and  to  staff  all  of  the  training  sessions. 

•  Worked  with  the  Aurora  Conference  Center  staff  and  online  meeting  space  reservation 
systems  at  two  locations  in  arrange  audio  visual  requirements  and  room  set  up  for  all  44 
training  sessions  at  two  sites. 

•  Collaborated  with  the  Learning  Connection  staff  to  generate  weekly  lists  of  nurses  enrolled 
in  the  training  sessions  and  communicate  unit-based  registration  data  to  monitor  progress  to 
the  Nurse  Leaders. 

GOAL  #6  Carry-out  the  intervention  study  at  the  ASLMC  site 

STATUS:  Completed  milestone  -  02-DEC-2014 

•  Optimization  Training  was  delivered  for  all  Nurse  Leaders  (45  -  100%).  4-SEPT-2014 

•  Optimization  Training  for  staff  nurses  started  on  9-SEPT-2014.  The  initial  plan  was  to 
utilize  breakout  sessions  to  promote  small  group  discussion  for  enhanced  learning.  The  first 
session  revealed  that  small  group  discussion  led  to  variations  in  content  delivery  with 
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challenges  to  time  management.  We  altered  the  training  plan  immediately,  reducing  the 
number  of  instructors  to  2-3  per  session  and  promoting  small  group  table  discussions  with  the 
larger  session  rather  than  breaking  out.  Logistical  adjustments  were  made  to  the  “Trainer 
Schedule”  and  meeting  room  reservations  to  accommodate  the  change. 

•  Staff  nurses  were  encouraged  to  use  the  Kronos  clock  at  the  training  center  to  enter  their 
hours  directly  into  the  Study  cost  center  for  optimal  accuracy  of  their  in-service  time. 

•  Collaborated  with  the  Learning  Connection  staff  to  update  the  tracking  system  with 
confirmed  attendance  for  documentation  and  dissemination  of  contact  hour  certificates. 

•  A  weekly  attendance  report  was  sent  to  Nurse  Leaders  so  confirm  attendance.  The  PM  and 
the  Post-Award  Grant  specialist  worked  closely  with  the  nurse  leaders  to  ensure  accurate  and 
timely  reimbursement  for  staff  nurse  training  participants. 

•  Optimization  Training  Session  evaluations  were  summarized  (Appendix  E) 

•  Optimization  Training  was  delivered  for  90%  (N=849)  staff  nurses  3  l-OCT-2014  with 
supplemental  training  sessions  for  unit  preceptors  by  study  group  (A  &  B)  to  review  the 
essential  practices,  their  role,  and  how  to  use  assigned  monitoring  strategy. 

GOAL  #7  Complete  tracking  process  of  the  intervention 

STATUS:  Completed  with  Post  Intervention  Assessment  20-JULY-2015 

•  Meetings  were  held  with  unit  leaders  to  review  results  with  audit  and  feedback,  to  identify 
priorities  and  to  discuss  ways  to  do  it  on  unit. 

•  Funds  were  provided  to  support  unit  leaders  to  engage  their  preceptor  staff  (2/unit)  to 
implement  the  essential  practices  by  identifying  priorities  and  working  to  improve  adopter 
skills  by  monitoring  and  providing  feedback  to  maintain  the  practices  on  the  unit. 

•  Created  and  distributed  Preceptor  Tracking  Tool  to  document  time  and  activities  conducted 
to  support  unit  implementation  30-NOV-2014 

•  Enlisted  the  study  site  “Magnet”  Program  Manager,  who  works  with  units  on  quality 
improvement  activities,  to  ensure  that  the  implementation  activities  aligned  with  established 
priorities  and  to  support  unit-based  nurse  leaders  and  preceptors  to  document  their  follow-up. 

•  Provided  informal  support  and  fonnal  meetings  with  unit  leaders  and  preceptors  to 
implement  and  maintain  essential  practices. 

•  Initiated  one-on-one  unit  follow  up  to  evaluate  Preceptor  skills  and  facilitate  Implementation 
activities,  involving  Clinical  Advisor/Data  Collector  (Marzinski)  31 -JAN-20 15 

•  Researcher  support  of  Unit  Implementation  activities  concluded  with  the  start  of  the  post¬ 
implementation  assessment  30-MARCH-2015 

•  Collected  preceptor  tracking  tools  to  document  time  and  activities  conducted  to  support  unit 
implementation  with  validation  of  time  reported  and  time  paid  by  grant. 

•  Compiled  and  completed  data  entry  of  preceptor  tracking  logs  15-SEPT-2015 

•  Collaborated  with  the  study  site  Magnet  Program  Manager  to  compile  Unit  Implementation 
and  outcome  data  30-SEPT-2015 

GOAL  #8  Complete  a  full  evaluation  measuring  the  impacts  of  KBN  methods  on  patient 

outcomes 

STATUS:  Post-implementation  assessment  is  complete;  Full  evaluation  in  progress 

•  Reviewed  baseline  data  collection  forms  and  made  minor  adjustments  identified  as  needed 
during  data  entry  (IRB  Modification  #9) 
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•  Created  a  master  schedule  for  post-implementation  assessment  for  23  units  at  the  study  site. 
Student  rotation  and  observer  schedules  were  consulted  and  all  team  calendars  were  blocked. 
Adjusted  schedule  to  accommodate  regulatory  visits. 

•  Produced  data  collection  materials  including  unit  information  packets  15 -MARCH-20 15 

•  Clinical  Advisors  distributed  packets  and  re-oriented  units  to  the  study  15-MARCH-2015 

•  Re-oriented  data  collection  team  with  updated  forms  17-MARCH-2015 

•  Oriented  new  Patient  Survey  data  collector  (Badger)  17-MARCH-2015 

•  Updated  the  Learning  Connection  module,  the  survey  and  recruitment  materials  for  the  Nurse 
Survey  (approved  by  the  IRB  -  Modification  #10)  1 4-MAY -2015 

•  Met  with  site  Clinical/Management  Advisors  to  plan  unit-level  recruitment  for  the  post¬ 
intervention  Nurse  Survey.  Utilized  same  process  for  supporting  staff  nurse  participation  on 
work  time  with  contributions  to  the  unit  Education  Fund  (45  minutes  of  average  staff  nurse 
salary  for  time  spent). 

•  Provided  active  nurse  survey  recruitment  support  including  weekly  participation  updates  to 
Unit  leaders 

•  Completed  post-implementation  assessment  including: 

o  Nonparticipant  (N=3 60)  and  admissions  observations  (N=4 8)  5-JUNE-2015 
o  Patient  surveys  (N=180)  5-JUNE-2015 
o  Nurse  Survey  (N=467)  20-JULY-2015 

■  Closed  Learning  Connection  access  and  downloaded  participant  rosters 

■  Finalized  participation  list  by  cross  checking  rosters  with  survey 
demographics  to  confirm  participation  and  ensure  grant  reimbursement  to 
units. 

■  Communicated  final  counts  and  ensured  transfer  of  appropriate  funds  to 
reimburse  units  for  staff  nurse  and  leader  time  spent  taking  the  survey 
($10,791). 

•  Completed  post-implementation  Nonparticipant,  Admission  and  Nurse  Leader  Observation 
data  entry  with  associated  chart  audits  with  auditor  reorientation  and  reliability  testing  -  25- 
SEPT-2015 

•  Cleaned  post  implementation  Nurse  Survey  data,  submitted  to  biostatistician  and  completed 
pre-post  analysis  NOV-2015 

•  Cleaned  post-implementation  Patient  Survey  data,  submitted  to  biostatistician  and  began  pre¬ 
post  analysis  DEC-2015 

•  Identified  unit  priority  outcomes,  completed  data  entry  and  cleaning  of  Unit  implementation 
and  outcome  data,  submitted  to  biostatistician  and  began  pre-post  analysis  JAN-2016 

•  Cleaned  post-implementation  Nonparticipant,  Admission  and  Nurse  Leader  Observation  data 
submitted  to  biostatistician  and  began  pre-post  analysis  FEB-2016 

•  Completed  detailed  analysis  plan  through  ongoing  meetings  with  biostatistician 

***************************************************************** 

No  Cost  Extension  -  Replication  of  Study  at  two  community  hospitals 

Four  additional  inpatient  units  from  Aurora  Memorial  Hospital  at  Burlington  (AMHB)  and 

Aurora  Lakeland  Medical  Center  (ALMC)  were  successfully  recruited  to  participate  in  the  study. 

Replication  Goals  #1-3 

Not  Applicable 
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Replication  Goal  #4:  Conducting  baseline  measurements  to  identify  gaps 
STATUS:  Completed  Milestone  14-OCT-2015 

•  Met  with  Chief  Nurse  Aurora  Lakeland  Medical  Center  (ALMC)  and  Aurora  Memorial 
Hospital  Burlington  (AMHB)  to  confirm  interest/support  (with  letter  for  IRB)  15-MAY- 
2015 

•  Revised  the  protocol  submitted  it  to  the  Aurora  IRB  as  a  modification  with  supporting 
documentation.  Obtained  IRB  approval  28-MAY-2015 

•  Submitted  letter  to  the  Department  of  Defense  Contracting  Officer  Representative  (COR) 
with  revised  the  Statement  of  Work  (SOW)  (available  by  request)  to  propose  a  no-cost  9- 
month  extension  to  replicate  the  study  at  two  smaller  sites  to  increase  generalizability.  8- 
JUNE-2015 

•  Updated  KBN  Study  recruitment  video  for  new  sites  (with  IRB  approval)  JUNE-2015 

•  Met  with  Chief  Nurse  and  Unit  Leaders  to  plan  recruitment  25-JUNE-2015 

•  Recruited  and  enrolled  4  inpatient  nursing  units  into  the  Study  based  on  study  protocol 
(30-JULY-2015) 

•  Created  a  master  schedule  for  baseline  assessment  for  4  units  at  the  new  study  sites. 
Student  rotation  and  observer  schedules  were  consulted  and  all  team  calendars  were 
blocked. 

•  Communicated  with  observation  schedule  to  units 

•  Met  with  and  oriented  Clinical  advisors  at  the  new  study  sites 

•  Created  and  distributed  orientation  packets  to  prepare  for  the  baseline  assessment 

•  Produced  all  observation  forms  and  posters 

•  Secured  space  including  printer,  phone  and  storage  at  the  new  study  sites. 

•  Prepared  Nurse  Survey  materials  for  survey  monkey  and  Learning  connection. 

•  Completed  Baseline  Assessment  including: 

o  Nonparticipant  (N=5 1)  and  admission  observations  (N=10)  25-SEPT-2015 
o  Patient  surveys  (N=27)  23-SEPT-2015 
o  Nurse  Survey  (N=51)  14-OCT-20 15 

•  Collaborated  with  leaders  to  ensure  accurate  reimbursement  for  Nurse  Survey  hours 

Replication  Goal  #5:  Design  the  Intervention  Study  strategy  including  the  delivery  method 
Not  Applicable 

Replication  Goal  #6:  Update  based  on  gaps  Deliver  the  intervention  (Optimization 
Training)  at  new  sites 

STATUS:  Completed  milestone  20-NOV-2015 

•  Cleaned  nurse  survey  data,  analyzed  results  and  identified  gaps  in  knowledge. 

•  Updated  audit/feedback  results  video  based  on  the  baseline  assessment 

•  Updated  training  module  content  to  address  gaps  in  knowledge  identified  in  the  baseline 
assessment. 

•  Planned  and  conducted  Optimization  Training  2-NOV-2015  to  20-NOV-2015 

o  Collaborated  with  Nurse  Leaders  regarding  logistics  for  Optimization  Training 
o  Drafted  Staff  Nurse  and  Nurse  Leader  Optimization  Training  schedule  to 
accommodate  Unit  participation,  Reserved  meeting  space 
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o  Submitted  Continuing  Education  application  with  approval 
o  Worked  with  Learning  Connection  to  create  courses 
o  Reviewed  and  prepared  updates  for  the  training  hand  out  materials  based  on 
baseline  findings 

o  Created  all  training  hand  out  materials 

o  Monitored  registration  progress,  provided  weekly  updates  to  Leaders 
o  Completed  training  sessions  for  leaders  (1  session),  (N=9,  100%)  and  staff  (13 
sessions),  (N=72,  76%) 

o  Reconciled  rosters  with  attendance  for  each  session  and  communicated  with 
leaders 

o  Provided  active  oversight  of  staff  to  ensure  accurate  Kronos  transfers 
o  Completed  evaluation  summary  for  WNA  continuing  education  credits 

Replication  Goal  #7:  Complete  Tracking  Progress  at  new  sites 
STATUS:  In  progress  (Scheduled  to  be  completed  by  25-APRIL-2016) 

•  Conducted  webinar  with  Nurse  Leaders  to  review  site -based  results  and  the  use  of 
electronic  reports  to  implement  and  maintain  essential  evidence-based  practices  13- 
NOV-2016 

•  Communicated  behavioral  expectations  for  unit  implementation,  e.g.,  identify  goals, 
preceptors  and  timing  for  additional  training  and  support 

•  Met  with  unit  leaders  to  further  review  baseline  results  and  to  identify  unit  priorities  for 
implementation  9-DEC-2015 

•  Scheduled,  planned  and  conducted  two  CNS/Preceptor  “super’  training  session  to  discuss 
roles  and  strategies  to  monitor  unit  implementation  and  ensure  follow  up  and 
maintenance  10-DEC-2015 

•  Communicated  weekly  with  CNSs,  Leaders  and  Preceptor  throughout  January  and 
February  to  provide  targeted  coaching  and  support  both  in  person  and  via  conference  call 
as  needed  to  ensure  implementation  strategies  are  being  deployed. 

Replication  Goal  #8:  Complete  Full  Evaluation  (Post-Implementation  Assessment  and 
Final  Report) 

STATUS:  Pending 

•  Completed  baseline  nonparticipant  and  admission  observation  data  entry  and  chart  audits 

•  Cleaned  baseline  patient  survey  data 

•  Initiated  unit  and  site-specific  nursing  sensitive  process  and  outcome  data  collection 

•  Created  a  master  schedule  for  post  implementation  assessment  (APRIL-MAY-2016)  for  4 
units  at  the  new  study  sites.  Student  rotation  and  observer  schedules  were  consulted  and 
all  team  calendars  were  blocked. 
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TRAVEL  AND  CONFERENCE  ATTENDANCE 


Year  1: 

•  PI  attended  2013  Midwest  Nursing  Research  Society  Conference-Chicago  to  recruit  intern 
(MAR-2013)  and  participate  in  preconference  session  on  Implementation  Research-  NOT 
funded  by  Grant 

•  PI  and  Graduate  Student  Intern  attended  the  Annual  AMIA  Informatics  Conference  (NOV- 
2013)  in  Washington  DC  to  network  and  participate  in  a  preconference  workshop  on  patient 
engagement 

•  Federal  Military  Advisory  and/or  TATRC  Review  -  Presentations  not  requested  -  Travel  was 
deferred. 

Year  2: 

•  Clinical  Decision  Support  (CDS)  consultant  traveled  to  Milwaukee  (May-2014)  to  meet  with 
the  team  and  observe  the  research  methodologies  in  progress. 

•  PI  collaborated  with  Graduate  Student  Intern  and  Consultant  to  submit  an  abstract  for  the 
Annual  AMIA  (Infonnatics)  Conference  in  Washington  DC  for  an  interactive  presentation 
on  Patient  Engagement  in  Acute  Care  -  it  was  not  selected  for  presentation.  PI  submitted 
details  about  KBN  Impact  Study  with  sponsor  information  for  a  report  presented  at  the 
AMIA  Nursing  Infonnatics  Working  Group  (NIWG)  Fall  Symposia  Event  (Sunday  16- 
NOV-2014);  PI  attended  annual  AMIA  conference  to  network  and  attend  sessions. 

•  Federal  Military  Advisory  and/or  TATRC  Review  -  Presentations  not  requested  -  Travel  was 
deferred. 

Year  3: 

•  PI  and  Research  Scientist  attended  the  Midwest  Nursing  Research  Society  (MNRS)  2015 
Annual  Research  Conference  -  17-APRIL-2015  -  Indianapolis,  IN  for  poster  presentation 

•  PI  and  Research  Team  Members  traveled  to  the_2015  Epic  User  Group  Meeting  (UGM)  -  2- 
SEPT-2015  -  Verona,  WI  for  two  podium  presentations 

•  PI  and  PM  traveled  to  Ft.  Detrick,  MD  for  the  In  Person  Review  (1 -DEC-20 15)  to  present 
progress  of  the  study 


KEY  RESEARCH  ACCOMPLISHMENTS 

1)  Institutional  Review  Board  and  Human  Research  Protections  Office  Review 

Protocol  Title:  “The  Impact  of  Electronic  Knowledge-Based  Nursing  Content  and  Decision- 
Support  on  Nursing-Sensitive  Patient  Outcomes” 

Approvals/Continuing  Reviews  to  Data: 

•  Aurora  IRB  Approval  #13-1 42E  20-Dec-2013:  Updated  with  Waivers  03-JAN-2014 

•  Aurora  IRB  Continuing  Review  of  Expedited  Study  (Exp  Cat  5  &  7):  Approved  24-NOV-2014 
and  11/24/15 

•  Human  Research  Protections  Office  (HRPO)  Approval  Log  No.  A- 17696  EDMS  #5648  21 -Feb- 
20 14;  Approved  Continuing  Review  documents  with  updated  protocol  acknowledged  19-DEC- 
2014  and  23-NOV-2015 
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Study  Protocol  Modifications: 

•  AHC  IRB  Modification  #1  -Ketchum  added  (completed  CITI  Training  &  Orientation)  - 
Approved  08-JAN-2014 

•  AHC  IRB  Modification  #2  -  Removing  Hartwig/Mills,  updating  *Patient  Survey  question 
with  review  of  Unit  Recruitment  Video  -  Approved  3  l-JAN-2014 

•  AHC  IRB  Modification  #3  -  Review  of  *observation/audit  forms  -  approved  03-FEB-2014 
*Note  -  Final/ AHC  IRB  approved  versions  of  the  Patient  Survey  and  Nonparticipant  and 
Audit  forms  were  forwarded  to  HRPO  prior  to  final  approval 

•  AHC  IRB  Modification  #4  -  Updated  HRPO  address  with  Patient  Consent  edits  requested  by 
HRPO-  Approved  28-FEB-2014 

•  Study  Protocol  and  AHC  IRB  approved  documents  with  HRPO  edits  were  reviewed  by 
Patricia  Shank,  CTR  for  US  ARMY  MEDCOM  for  review 

•  Study  Protocol  submitted  to  HRPO  -  Dr.  Laura  R.  Brosch,  RN,  PhD,  Director  of  the  Human 
Research  Protection  Office  (HRPO)  Office  Approved  21-FEB-2014 

•  Aurora  IRB  acknowledged  received  of  HRPO  Approval  Letter-  24-FEB-2014 

•  AHC  IRB  Modification  #5  with  final  Nurse  Survey  with  Learning  Connection  ppt  to 
introduce  survey  -  Approved  25-MAR-2014 

•  AHC  IRB  Modification  #6  to  add  Interim  RN  Data  Collector  (completed  CITI  Training  and 
Orientation)  with  updated  PI  Address  (moved  to  new  location)-  Approved  10-APR-2014 

•  AHC  IRB  Modification  #7  to  add  Project  Manager  (Nikolic)  and  Research  Scientist 
(Badger)  (completed  CITI  Training/Orientation)  with  revised  fliers  for  Nurse  Survey 
recruitment-  Approved  03-JUN-20 14 

•  AHC  IRB  Modification  #8  with  updated  IRB  Document  and  Study  Protocol  (Version  #3  - 
06-NOV-2014)  with  updated  conceptual  framework  and  details  about  randomization  plan, 
optimization  training,  and  unit  implementation.  The  Nurse  Infonnation  Letter  and  Patient 
Consent  fonns  were  updated  with  PI  address  change.  Added  Research  Scientist  (Martens) 
(completed  CITI  Training/Orientation)  to  replace  Bauer  (resigned)  Approved  20-NOV-2014 

•  AHC  IRB  Modification  #9  with  updated  Study  Protocol  (Version  #4  -  09-FEB-2015)  with 
editorial  changes  to  enhance  background  and  process  description,  updated  observations  and 
audit  fonns  to  capture  data  that  were  written  in  during  baseline  assessments,  updated  fliers 
and  other  study  materials  with  revised  PI  contact  information,  updates  to  the  Nurse  Survey 
Learning  Connection  module  in  preparation  for  use  during  post-intervention  assessment. 
Updated  role  for  research  scientist  (Badger)  to  include  recruitment,  consent,  and  data 
collection  of  Patient  Survey  (in  place  of  Bauer)  and  added  clinical  advisor  (Marzinski  -  Site 
Magnet  Coordinator)  for  limited  hours  to  support  tracking  of  unit-based  implementation 
including  preceptor  staff  use  of  maintenance  tools  and  tracking  form.  Approved  15-Feb-2015 

•  AHC  IRB  Modification  #10  with  updated  flier  and  email  to  introduce  the  Nurse  Survey 
Approved  14-May-2015 

•  AHC  IRB  Modification  #11  with  updated  Study  Protocol  (version  #5  -  28-MAY-2015) 
proposing  the  addition  of  two  small  community  sites  to  increase  the  diversity  of  our  sample 
and  broaden  generalizability.  Proposed  to  add  four  additional  inpatient  nursing  units,  90 
nurses  and  80  patients.  Also  updated  recruitment  video  script  and  Nurse  Information  Letter  to 
include  infonnation  about  the  two  new  sites.  Approved  28-MAY-2015 

•  AHC  IRB  Modification  #12  with  Personnel  Changes  (Resignation  Badger/Rehire  Bauer) 
submitted  9-OCT-2015 


18 


Award  #:  W81XWH-1 3-1 -0034 
Aurora  Health  Care  -  Year  3  Report 


2)  Updated  Theoretical  Framework  and  Intervention  Plan  for  the  Study 

The  KBN  Team  reviewed  the  literature  and  selected  the  Dissemination  of  Evidence-Based  Policy 
framework  (Appendix  B)  by  Dodson,  Brownson,  and  Weiss  (2012,  p.  440)  to  explain  how 
context  and  dissemination  and  implementation  strategies  influence  the  adoption  and  use  of  the 
KBN  innovation.  Although  the  model  is  conceptualized  for  public  health,  the  concepts  appeared 
to  be  relevant  to  the  evidence-based  policy  process  used  in  acute  care.  Dodson  and  colleagues 
(2012)  describe  three  key  domains  to  implement  evidence-based  practice:  policy  content,  policy 
process,  and  policy  outcomes.  “Policy  content”  focuses  on  identifying  the  specific  evidence 
based  policy  elements  that  are  likely  to  be  effective.  “Policy  outcomes”  refers  to  the  overall 
effect  of  policy  implementation.  “Policy  process”  refers  to  the  many  factors  including  the 
structure  and  scope  of  the  process,  the  presence  and  standing  of  the  policy  “sparkplug” 
(facilitator)  and  their  ability/skills  to  articulate,  advocate,  and  communicate  support  for  the 
policy.  Leaders  can  choose  a  “dissemination”  (passive)  approach  to  increase  target  audience 
“awareness”.  They  can  also  choose  an  “implementation”  (active)  approach  with  active  strategies 
that  facilitate  the  adoption,  implementation,  and  maintenance  processes.  “Adoption”  is  defined 
as  “a  decision  to  make  full  use  of  an  innovation  as  the  best  course  of  action  available”  and  to 
take  steps  to  identify  and  address  barriers  to  adoption.  “Implementation”  refers  to  ‘the  extent  to 
which  an  innovation  is  carried  out  with  completeness  and  fidelity”  with  a  focus  on  improving  the 
skills  of  the  adopters  through  training  and  technical  assistance.  “Maintenance”  refers  to  the 
extent  to  which  an  innovation  becomes  embedded  into  the  nonnal  operation  and  maintained  by 
policy  enforcement  (p.  440). 

The  PI  participated  in  National  Institutes  of  Health  (NIH)  Training  Institute  for  Dissemination 
and  Implementation  Science  (TIDIRH)  (Refer  to  Additional  Achievements).  The  institute 
provided  education  and  guidance  for  adapting  the  Dobson  et  ah,  (2012)  conceptual  framework 
for  use  with  nurses  in  acute  care.  We  received  advice  on  how  to  redesign  the  implementation 
intervention.  The  international  experts  (including  Dr.  Ross  Brownson  -  an  author  of  the  Dodson 
et  al.  model  and  Dr.  Sharon  Straus)  supported  the  conceptual  framework  adaptation  and  advised 
us  to  be  more  specific  about  the  core  components  of  the  KBN  Innovation  (Appendix  B)  and  to 
utilize  a  multifaceted  implementation  strategy.  They  also  encouraged  us  to  use  a  head  to  head 
comparison  of  maintenance  strategies  rather  than  using  a  “control”  group  with  dissemination 
since  dissemination  by  itself  is  not  effective. 

The  adapted  theory  (Appendix  C)  supports  the  evaluation  of  both  hypotheses  in  this  study  with 
the  addition  of  two  intermediary  outcomes:  EBP  knowledge  and  use  of  EBP  behaviors  as  an 
essential  step  to  achieve  optimal  outcomes.  These  intermediary  outcomes  are  not  commonly 
evaluated  in  EBP/knowledge  translation  implementation  research  (Yost,  Ganann,  Thompson, 
Aloweni,  Newman,  et  al.,  2015),  but  measurement  of  these  factors  may  provide  important 
information  to  help  us  to  understand  why  EBP  uptake  is  limited.  Dissemination  refers  to  passive 
knowledge  transfer.  Implementation  refers  to  a  more  active  approach  influenced  by  the  unit 
environment  and  the  degree  to  which  EBP  is  adopted,  implemented  and  maintained.  The 
adapted  model  describes  the  importance  of  “adoption”,  having  clear  behavioral  expectations, 
assessing/  improving  adopter  skills  and  the  use  of  maintenance  strategies  to  ensure  use.  Dodson 
and  colleagues  also  indirectly  referred  to  the  use  of  a  “sparkplug”  (facilitator)  to  achieve  and 
maintain/enforce  use  of  the  evidence-based  process  but  offered  no  specific  details  about  this  role 
in  the  model.  Our  baseline  findings  suggested  that  nurses  leaders  were  not  often  observed  giving 
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feedback  to  staff  regarding  the  phenomena  in  focus  for  this  study.  All  of  the  study  sites  utilize 
staff  nurses  who  intennittently  serve  in  the  role  of  “preceptor”  with  responsibilities  for  role 
modeling,  educating,  and  socializing  new  staff.  We  asked  nurse  leaders  to  identified  (2) 
preceptors  to  support  them  to  implement  evidence-based  practices  on  their  unit. 

The  study  intervention  was  revised  to  be  conceptually  consistent  with  the  theory,  delivering  a 
multi-modal  implementation  strategy  to  support  unit-based  implementation  and  maintenance. 
Implementation  refers  to  the  “process  of  putting  to  use  or  integrating  evidence-based 
interventions  within  a  setting”  (Rabin,  Brownson,  Haire-Josu,  Kreuter,  Weaver,  2008,  p.  118). 
Researchers  have  reported  that  multifaceted  implementation  strategies  are  more  effective  in 
supporting  nurses  to  use  evidence-based  practices  during  patient  care  (Ista,  van  Dijk,  van 
Achterberg,  2013;  Ivers,  Jamtvedt,  Flottorp,  et  al.,  2012,  Matthew-Maich,  Ploeg,  Dobbins, 

Jack, 20 13;  Wuchner,  2014). 

Commonly  used  implementation  strategies  include  (1)  embedding  the  practices  into 
organizational  structures  such  as  policies  and  documentation  (Matthew-Maich,  et  al.,  2013),  (2) 
audit/feedback,  (3)  educational  meetings  with  clear  behavioral  expectations,  and  (4)  leadership 
strategies  to  improve  and  maintain  adopter  skills  over  time  (Wuchner,  2014).  Audit  and 
feedback,  defined  as  “a  summary  of  clinical  performance  over  a  specified  period  of  time”,  is 
used  to  prompt  professionals  to  modify  their  practice  behavior  to  be  consistent  with  established 
standards  (Ivers,  et  al.,  2012),  In  a  systematic  review,  audit  and  feedback  as  an  implementation 
strategy  was  associated  with  small  (4.3%)  but  potentially  important  improvements  in  practice, 
and  was  most  effective  when  baseline  performance  is  low,  when  given  by  a  supervisor  or 
colleague,  provided  more  than  once,  delivered  in  verbal  and  written  fonnats,  and  with  explicit 
targets  for  improvement  (Ivers  et  al,,  2012).  Educational  strategies  had  a  small  effect  (Cheater, 
Baker,  Reddish,  Spiers,  Wailoo,  et  al.,  2006)  at  6  months  but  most  studies  report  that  educational 
strategies  by  themselves  were  not  insufficient  to  achieve  outcomes  (Ista,  et  al.,  2013;  Thompson, 
Estabrooks,  Scott-Findlay,  Moore,  Wallin,  ,  2007;  Wuchner,  2014). 


3)  Current  State  Literature:  EBP  Use  by  Nurses  with  Role  of  Nurse  leader 

Researchers  have  historically  reported  that  limited  resources,  knowledge,  skills,  and  time  posed 
barriers  to  the  use  of  EBP  by  nurses  in  hospitals  (Hannes,  et  al.,  2007;  Pravikoff,  et  al.,  2005).  In 
their  recent  integrative  review,  Sanders  &  Vehvilainen-Julkunen  (2016)  found  that  the  vast 
majority  (81%)  of  research  studies  on  nurse  readiness  for  EBP  were  descriptive  cross-sectional 
surveys  evaluating  attitude  and  self-reported  knowledge,  skills,  and  use  of  EBP.  They  concluded 
that  more  robust,  theoretically-based,  and  psychometrically  sound  nursing  studies  were  needed  to 
test  and  evaluate  the  effectiveness  of  interventions  to  advance  the  EBP  competencies  of  nurses. 

Researchers  continue  to  report  that  the  use  of  EBP  in  clinical  settings  by  RNs  remains  low 
(Duffy,  Culp,  Yarberry,  Stroupe,  Sand-Jecklin,  et  al.,  2015;  Melnyk  et  al.,  2012;  Yoder  et  al., 
2014).  Most  recently,  Melnyk  and  colleagues  (2016)  reported  the  findings  of  a  national  survey 
asking  chief  nurse  executives  (CNEs)  about  EBP  and  outcomes  at  their  facilities.  Many 
hospitals  reported  low  scores  on  key  nurse  sensitive  perfonnance  indicators.  Many  of  the  CNEs 
reported  high  value  for  EBP,  but  reported  low  EBP  implementation  by  nurse  leaders  and  point- 
of-care  clinicians.  Safety  and  quality  were  high  priorities.  EBP  was  a  low  priority  with  limited 
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budget  allocation.  Warren  and  colleagues  (2016)  also  reported  leader  barriers  to  the  integration 
of  EBP  that  was  associated  with  leader  turnover  and  competing  priorities.  Melynk  (2016)  made 
an  urgent  plea  for  nursing  organizations  to  provide  top-level  support,  role  modeling,  and 
monetary  investment  to  implement  EBP  and  drive  improved  outcomes.  Engaging  leaders  (and 
informal  opinion  leaders)  has  been  shown  to  have  a  positive  effect  on  EBP  knowledge,  attitudes, 
and  behaviors  within  the  community  (Flodgren,  Pannelli,  Doumit,  et  ah, 201 1;  Gifford,  Davies, 
Graham,  Tourangeau,  Woodend, ,  et  al.,  2013;  Huis,  et  al.,  2013;  Park,  Zafran,  Steward,  et  al 
2014;  Stetler,  Ritchie,  Rycroft-Malone,  Charns,  2014;  Valente  &  Pumpuang,  2007  ;Yost,  et  al., 
2015).  Stetler  and  colleagues  (2014)  described  that  leaders  must  be  engaged  in  functional, 
strategic,  and  cross-cutting  behaviors,  including  deliberately  and  routinely  talking  about  and 
demonstrating  desired  EBP  behaviors.  Gifford  and  colleagues  (2013)  reported  that  current  nurse 
leader  practice  did  not  routinely  involve  monitoring  adherence  to  guideline-based  care  but  this 
may  be  an  effective  strategy  to  be  present,  communicate,  and  reinforce  responsibilities.  Tools 
may  be  needed  to  help  leaders  to  monitor  and  know  if/when  best  practices  were  used  in  daily 
practice  (Gifford,  et  al,  2013;  Ista,  et  ah,  2013;  Matthew-Maich,  et  ah,  2013). 

Limitations 

This  study  represents  a  comprehensive  assessment  of  the  context,  processes,  and  outcomes 
associated  with  the  use  of  a  technological  innovation  designed  to  support  nurses  to  know  and  use 
best  practices  in  patient  care.  This  study  was  conducted  using  all  inpatient  nursing  units  within  a 
single  health  care  system.  This  study  is  designed  to  be  conducted  within  a  single  organization 
because  the  customized  build  is  not  available  at  any  other  facilities.  This  single-site  design  limits 
generalizability,  but  is  not  unusual.  Buntin  and  colleagues  (2011)  analyzed  HIT  research  and 
found  that  more  than  half  (64%)  of  the  studies  in  their  review  were  conducted  in  a  single 
institution  or  tightly  integrated  network.  Replication  of  this  study  at  two  additional  facilities  that 
vary  in  size,  leadership  structure,  and  Magnet  status  will  add  diversity  to  the  sample  and  enhance 
the  overall  generalizability. 

CONCLUSION: 

The  “KBN  Impact  Study”  represents  a  robust,  theory-based  study  evaluating  the  impact  of 
technology  on  nursing  knowledge,  behaviors  and  patient  outcomes  at  three  diverse  study  sites. 
The  study  is  proceeding  toward  goal  accomplishment  according  to  plan  and  under  budget.  The 
Research  Team  has  reviewed  the  literature  and  networked  with  international  leaders  in 
dissemination  and  implementation  research  to  build  a  strong  theoretical  foundation  for  the  study. 
The  study  site  was  solidly  engaged  in  participating  in  the  study  as  evidenced  by  100%  of 
inpatient  units  recruited  for  the  study,  high  (49%)  voluntary  response  rate  to  the  Nurse  Survey, 
and  high  (100%  Leader  and  90%  of  Staff  Nurse)  participation  rate  in  the  Optimization  Training. 
We  have  created  useful  tools  and  gathered  an  extensive  amount  of  real-world  qualitative  and 
quantitative  data  with  analysis  and  hypotheses  testing  in  progress.  We  are  also  utilizing  unused 
funding  to  replicate  the  study  (no-cost  extension  of  this  grant)  in  a  smaller,  community  site  to 
increase  generalizability  of  the  findings.  We  look  forward  to  completing  the  post-intervention 
assessment  and  sharing  results  of  this  important  work  over  this  our  final  year. 
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PUBLICATIONS,  ABSTRACTS,  AND  PRESENTATIONS 
Publications  -  Manuscripts  in  progress 

•  KBN  Impact  Study  Protocol  Paper  for  journal:  Implementation  Science 

•  Theory  Paper  re:  the  adapted  Conceptual  Framework  for  the  Dissemination  and 
Implementation  of  Evidence-based  Policy 

Journal:  Journal  of  Advanced  Nursing 

•  Methodology  Paper  re:  Mixed  Methods  Study  for  Baseline 
Journal:  Computers,  Informatics,  Nursing  (CIN) 

Abstracts  (not  accepted/pending) 

•  Abstract  submitted  to  AMIA  Conference  (MAR-2014)  for  Fall  2014.  Preliminary  findings 
were  in  progress  so  we  did  not  have  data  to  qualify  as  a  research  submission.  The  Research 
Team  collaborated  with  Dr.  Dowding  (consultant)  and  another  colleague  to  submit  a 
proposal  for  an  interactive  panel  presentation  on  Patient  Engagement  in  Acute  Care. 

Results:  Not  accepted 

•  Abstract  submitted  to  AMIA  Conference  (MAR-20 1 5)  for  fall  2015 

Title:  “The  Impact  of  Electronic  Knowledge-Based  Nursing  (KBN)  Content  and  Decision- 
Support  on  Nursing  Knowledge  and  Use  of  Evidence-based  Practices” 

Authors:  Hook,  Badger,  Gentile,  Giannini,  Hoffmann,  Ketchum  &  Martens 
Results:  Not  Accepted 

Presentations: 

•  Poster  Presentation:  “Measuring  the  Impact  of  Evidence-based  Patient  Education  on 
Patient  Knowledge  and  Behavior  in  Acute  Care”  Authors:  Bauer  &  Hook  at  the  16th  Annual 
Southeast  WI  Building  Bridges  Research  Conference  -  09-MAY-2014  -  Milwaukee,  WI 

•  Poster  Presentation:  “Evaluating  the  Impact  of  Evidence-Based  Patient  Education  on 
Patient  Knowledge  and  Behavior  in  Acute  Care”  Authors:  Bauer  &  Hook  at  the  Annual 
Aurora  Scientific  Day  Conference  -  21-MAY-2014  -  Milwaukee,  WI 

•  Poster  Presentation:  “Using  Implementation  Theory  to  Evaluate  the  Impact  of  Technology 
to  Support  Evidence-Based  Nursing  Practices  in  Patient  Outcomes  in  Acute  Care.”  Authors: 
Hook  &  Badger  at  the  Midwest  Nursing  Research  Society  (MNRS)  2015  Annual  Research 
Conference  -  17-APRIL-2015  -  Indianapolis,  IN 

•  Podium  Presentation:  “Using  Implementation  Theory  to  Evaluate  the  Impact  of 
Technology  to  Support  Evidence-Based  Nursing  Practices  in  Patient  Outcomes  in  Acute 
Care.”  Authors:  Hook  at  the  17th  Annual  Southeast  WI  Building  Bridges  Research 
Conference  -  8-MAY-2015  -  Milwaukee,  WI 

•  Poster  Presentation:  "The  Impact  of  Electronic  Knowledge-Based  Nursing  (KBN)  Content 
and  Decision-Support  on  Nursing  Knowledge  and  Use  of  Evidence-based  Practices", 
Authors:  Hook  &  Badger  at  the  Annual  Aurora  Scientific  Day  Conference  -  20-MAY-2015, 
Milwaukee,  WI 

•  Podium  Presentation:  “Using  Implementation  Theory  to  Study  How  Technology  Supports 
Best  Practice.”  Authors:  Hook,  Giannini,  Ketchum,  Hoffmann  at  the  2015  Epic  User  Group 
Meeting  (UGM)  -  2-SEPT-20 1 5  -  Verona,  WI 
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•  Podium  Presentation:  “Investigating  Adherence  to  Evidence-Based  Practice  at  the 
Bedside”  Authors:  Hook,  Giannini,  Ketchum,  Hoffmann  at  the  2015  Epic  User  Group 
Meeting  (UGM)  -  2-SEPT-2015  -  Verona,  WI 

•  Podium  Presentation:  “Using  a  Mixed  Methods  Design  to  Investigate  Adherence  to 
Evidence-Based  Nursing  Practices  in  Acute  Care”  Authors:  Hook  at  the  MNRS  40th  Annual 
Research  Conference  -  18-March-2016  -  Milwaukee  WI 

•  Keynote  Address:  “Nursing  at  a  Crossroad:  Is  Evidence-Based  Practice  Core  ...  or  NOT?” 
Authors:  Hook  at  the  18th  Annual  Southeast  WI  Building  Bridges  Research  Conference  -  13- 
MAY-20 1 6  -  Milwaukee,  WI 

INVENTIONS,  PATENTS  AND  LICENSES  -  None 

REPORTABLE  OUTCOMES:  (In  progress) 

•  Specifications  of  the  standardized  content,  CDS  tools,  electronic  care  plans,  and  report  tools 
in  the  EHR  are  available  for  use  in  scaling  the  project  for  the  future 

•  Team  is  collaborating  with  Epic  (vendor)  and  members  of  the  Epic  Nursing  Collaborative 
efforts  to  standardize  the  content  and  care  planning  tools  for  key  nursing  phenomena. 

•  Adapted  conceptual  framework  (Dodson,  et  ah,  2012)  used  for  the  study  with  input  from 
international  experts  in  Dissemination  and  Implementation  Research.  A  manuscript  to 
publish  the  KBN  Impact  Study  protocol  with  the  adapted  framework,  core  concepts,  and 
specifications  for  comparing  two  implementation  interventions  is  in  progress  for  submission 
to  the  journal  Implementation  Science. 

•  Methodology  paper  describing  the  key  aspects  of  this  mixed  methods  study  is  in  progress 

•  Lesson  Plans  for  training  staff  nurses,  nurse  leaders,  and  preceptors  regarding  the  essential 
evidence-based  practices  can  be  used  to  scale  project  for  the  future. 

•  Lesson  Plans  and  tracking  tools  for  training  preceptors  can  used  to  scale  project  for  the  future 


OTHER  ACHIEVEMENTS 

2014  Training  Institute  for  Dissemination  and  Implementation  Research  in  Health 

The  Principal  Investigator  submitted  an  application  for  the  2014  Training  Institute  for 
Dissemination  and  Implementation  Research  in  Health  (TIDIRH;  July  20-25,  2014)  -  an 
intensive  5  day  training  seminar  to  support  researchers  in  designing  and  conducting 
dissemination  and  intervention  (D  &  I)  research.  Meissner  and  colleagues  (2013)  authored  a 
paper  about  the  emerging  science  of  D  &  I  with  details  about  the  n.  The  2014  training  institute 
(4th  cohort)  was  sponsored  by  Harvard  University  and  the  Dana-Farber  Cancer  Institute  with 
support  from  the  National  Institutes  of  Health,  and  the  U.S.  Department  of  Veterans  Affairs.  Dr. 
Hook  was  one  of  41  participants  selected  from  a  pool  of  289  international  applicants  who  met  the 
credentialing  criteria  and  proposed  a  feasible  D  &  I  project  to  work  on.  The  KBN  Impact  Study 
protocol  and  intervention  plan  was  reviewed  by  national  and  international  experts  who  confirmed 
the  appropriateness  of  the  conceptual  framework  adaptation  (Appendix  C)  and  the  Intervention 
Plan  (Appendix  E)  with  audit/feedback,  training,  unit  implementation)  and  varied  maintenance. 
The  faculty  also  offered  support  to  the  PI  for  publication  of  the  study  protocol  and  theory  paper 
that  were  developed  as  a  result  of  the  consultation. 
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Federal  Military  Study  Advisory  Council  Collaboration 

Year  1: 

•  Established  a  collaborative  relationship  with  LTC  Michael  Ludwig,  RN-BC,  MS,  CPHIMS, 
AMEDD  Chief  Nursing  Information  Officer,  Ollie  B.  Gray  RN,  MSN,  PMP  Executive 
Healthcare  Manager,  AITG  for  TATRC  and  members  of  the  Federal  Nursing  Infonnatics 
iEHR  Collaborative 

•  Orientation  meeting  (conf  call)  held  with  LTC  Ludwig  and  associates  -  MAR-2013 

•  LTC  Michael  Ludwig  set-up  kick-off  /orientation  meeting  with  DOD  Nursing  Infonnation 
iEHR  Collaborative  Meeting  -  28-MAY-2013 

•  Worked  with  LTC  Michael  Ludwig  to  plan  subsequent  meetings  (July  31  2013)  with  plan  to 
draft  a  Council  Charter 

•  F/u  call  (30-AUG-  2013)  with  Federal  Military  Advisory  Committee.  Each  branch  has  their 
own  path/approaches  to  evidence  based  practice.  Discussed  need  for  determining  how  similar 
or  different  military  facilities  are  from  study  site. 

•  The  Assessment  form  was  developed  and  distributed  to  assess  processes  used  by  each  branch 
to  support  evidence-based  practice  during  the  SEPT-2013  meeting. 

•  Navy  Branch  Meeting  held  on  23-OCT-2013,  led  by  Captain  Joel  Parker  to  discuss  the  KBN 
research  project  to  attendees  and  request  input  from  Navy  Nurses.  The  Navy  representatives 
discussed  where  they  were  in  building  their  documentation  system  with  best  practices  and 
associated  protocols,  etc.  for  cross  military/cross  discipline  use.  Consensus  was  achieved 
around  the  need  for  strategies  to  ensure  adoption  and  evaluation  re:  infonnatics  build  to 
ensure  that  it  was  working  and  supporting  the  staff  to  effectively  achieve  outcomes.  The  call 
ended  with  shared  interest  but  uncertainty  regarding  next  steps. 

•  December  check-in  conference  call  was  held  13-DEC-  2013  with  Federal  Advisory  Council. 
To  date,  however,  none  of  the  branch  stakeholders  completed  the  Assessment  Tool  for 
gathering  information  about  the  nursing  structure  and  where  they  are  with  doing  evidence- 
based  practice  projects  supported  by  infonnatics.  (Assessment  deferred) 

Year  2 

•  The  KBN  Research  Team  continued  a  collaborative  relationship  with  our  Federal  Military 
Advisory  Council  led  by  LTC  Michael  Ludwig,  RN-BC,  MS,  CPHIMS,  Officer  in  Charge  to 
the  Presidential  Medical  Evaluation  Treatment  Unit  -  OIC  METU  and  the  members  of  the 
Federal  Nursing  Infonnatics  iEHR  Collaborative. 

•  Conference  Calls  were  held  quarterly  to  update  the  group  regarding  study  progress  including 
24-J AN-20 14  to  describe  recruitment 

7-APR-2014  to  describe  baseline  data  collection 
15-AUG-2014  to  describe  baseline  results  and  intervention  plan 

The  group  discussed  the  ways  in  which  this  research  can  provide  relevant  infonnation  regarding 

issues  that  are  facing  the  participating  agencies.  LTC  Seeley  suggested  study  could  be  presented 

at  the  2015  Defense  Health  Information  Technology  Symposium. 

Year  3 

•  The  collaboration  with  the  Federal  Military  Advisory  Council  continued  in  2015 

•  Advisory  group  fonnal  conference  call  took  place  on  20-APRIL-2015.  PI  presented  details 
about  the  active  intervention  and  plans  for  reassessment 
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•  The  military  advisory  group  discussed  the  status  of  the  military’s  efforts  to  select  a  new  EHR 

•  Advisory  members  were  still  unclear  about  the  link  between  the  study  and  their  work. 

•  A  planning  call  took  place  with  LTC  Ludwig  18-NOV-2015.  He  reported  his  role  had 
changed  and  was  unclear  regarding  the  link  between  the  study  and  the  work,  suggested  that 
we  seek  advice  from  our  COR. 

In  year  4,  researchers  will  communicate  with  Federal  Advisory  Board  members  to  look  for 

opportunities  to  present  findings. 
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APPENDIX  A.  KBNI  Conceptual  Framework,  Logic  and  Patient  Engagement  Models 
(Unchanged) 
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Conceptual  Model:  Using  Evidence-based  Interactions  to  Engage  Patients 
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APPENDIX  B.  Knowledge-based  Nursing  (KBN)  Innovation  Core  Concepts 


Core  Components: 

1)  KBN  is  a  chartered  entity  within  the  professional  nursing  infrastructure  at  the  study 

institution  with  goals  of  contributing  evidence-based  recommendations  to  the  system  nursing 
practice  council  for  embedding  into  the  content  of  nursing  policy  and  the  EHR.  The  KBN 
Department  is  also  responsible  for  governing  the  nursing  content  and  workflows  in  the  EHR. 


2)  KBN  Evidence  Summaries  focus  on  the  independent  role  of  the  nurse: 
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manage  problem,  and  engaging  patient  and  family  to  support  self-management  (know 
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Outcome  evaluation  for  achievement  by  the  close  of  the  inpatient  stay 


POC  Document  and  Synthesis  Document  starting  with  Table  of  Contents  (Sample) 


^nonkdgc-Risrd  Nursing 

Post-Fall  Care  in  Adults  in  Acute  Care 

Phenomenon  of  Concern  Document 

3.13.07.  SmUad  2008.  2014 
iurvra  Knowledge  Development  Team 

Llement 

Conn  in  for  l'ln  norm  non  or  C  oncern 

e 

Phenomenal  of 

concern  (PoC) 

Fan*-  Poa  Falls  Care 

i  6 

a  1  a  up  eyoup 

Infant  Child  {dropped),  Children  Age  >  1  veari  to  Adultt  i  tails,) 

s 

3  |  Condition 

Hospitalised  All 

1  Venue 

Acute  Care  Inpatient  and  Outpatient 

Definmeo 

A  Patient  Fad is  a  nursing  sensitive  qualitv  indicator  defined  as  an 
unplanned  descent  to  the  floor  (or  rxiraacoi  of  the  floor  e  g  trash  can  or 
other  equipment)  during  the  coiusc  of  a  palical  >  hospital  tlav  with  ue 
without  intury  to  tlie  patient,  and  occurs  on  an  eligible  repotting  nursing 
unit  {National  Database  for  Nursing  Quality  Indicators  [NDNQ1J.  201 » 
p  7  -  published  guideline) 

ftnow  tedge-Based  Nursing 

Risk  for  Falls  in  Acute  Care 
Synthesis  Document 
its o?  :oos  :o:4 

Auioca  Knowledge  Development  Turn 


r.f. 

Assessment  Keeommendatsons 

1  Screen  all  adulr  nanmts*  far  the  probable  iodicatoes  of  fall  risk  using  assessment  oitena 
on  silnutuea 

'Proceed  to  Assessment  Recommendation  •*  if  unable  to  rehably  screen  for  risk  or  if 
patient  is  under  age  IS  since  screening  is  not  recommended  for  pediatric  patients 
because  nsk  factors  arc  not  the  tame  at  adults 

3 

2  Screen  all  patients  Call  ages)  for  one  or  more  fall  related  rniuxv  risk  factors  {patient 
history  medication  list  and  laboratory  results)  on  admission 

7 

3  Screen  til  patterns  fos  special  conditions  that  mas  lead  to  imps  edict  cd  falls  dueto 
physiological  reasons 

10 

3)  POC-specific  content  and  clinical  decision  support  tools  in  the  EHR 

•  Standardized  assessments  including  reliable/valid  assessment  tools 


Doc  F  lowaheets 
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Daily  Caies/Safety 

Irtrawp  Lines 

tntake/Oulput 

Additional  Screenings 
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Pam  Assessment 
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Denial  Status 
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p 
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Gastroimasarvai 
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p 
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UuscuioskelMaJ 
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vf  Current  P»n  Assessment 
Pam  Assessment  Frequency 


vf  Mental  Status  Assessment  j 
Symptoms  of  DeAnum 
Vl  Mental  Status  Additional  Parametd| 
Mental  Status  Assessment  Frequency 


W|  Neurolopcal  Assessment 
4  V|  Additional  Parameters  {| 

Neurological  Assessment  Frequency  S 


03/30/11  0900 

Evidence-based  assessment  tools 
Standardized  nurse-based  content  with 
“Charting  by  Exception"  to  established  norms 
Patient-specific  assessment  plan 
“Tube"  data  Integrated  within  assessment 
Reference  Text  provided 
Navigators  and  monitoring  flowsheets 
“Shared"  rows  for  optimal  efficiency 


v|  Gastrointestinal  Assessment 
Gastrointestinal  Assessment  Frequency 

n 


£ 


V|  Gemtounnary  Assessment 
Gendounnary  Assessment  Frequency 
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•  Designing  evidence-based  care  planning  processes  to  fit  into  the  nursing  workflow 

-  Renewing  the  focus  on  planning  and  evaluating  patient  care 

-  Creating  clinical  decision-support  (CDS)  tools  to  identify  risks  and  actual  problems 

-  Designing  care  plans  to  plan  care  and  link  with  flowsheets  to  “associate” 
documentation  of  patient  status  and/or  interventions  with  the  plan  of  care. 


Y  Patient  has  symptoms  of  Delirium.  Best  evidence  recommends  urgent  provider  [elaboration  to  confirm  the  diagnosis  of  delirium  and  collaborate  to 
Identify  potential  causes  amid  initiate  interventions  to  address  causes  and  limit  the  severity  and  duration  of  the  event.  Initiate  Delirium  Care  Plan 
include  both:  Risk  lor  and  (actual)  Delirium  template  if  not  already  In  place. 

r  Deiinum  A  Delirium  (Actual)  Outcomes  are  similar  to  Risk  Plan 

1  ^Symptoms  of  deiinum  resold  fot  24  hours  a  - . - 

T  *  f  amifyi'patient  vertiair2es  understanding  of  deiinum  symptoms,  management,  and  followup  3i 
r  Problem  interventions 

r  t  Collaborate  with  provider  to  confirm  non-ETOH  delirium  diagnosis  and  identify  potential  causes  that  require  interventions  Si 

T  #  Collaborate  with  provider  if  symptoms  do  not  resotve  or  worsen  [J 
r  Implement  delirium  management  strategies  Si 
r  t  Implementfmamtam  eaily  mobilization  Si 

r  »  Ask  familyfcaiegrver  to  bnng  in  personal  and  familiar  cbieds  to  assist  in  orientation  and  normalization  If  needed  Si 
r  f  Collaborate  with  provider  rf  symptoms  of  fluid  imbalance  or  if  electrolyte  levels  aie  outside  of  established  parameters  Si 

r  f  tCU:  Collaborate  with  provider  to  minimize  the  duration  of  intubation  If  present  @[ 
r  Collaborate  with  physician  regarding  post-discharge  follow-up  to  ensure  recovery  and  to  identify  r 

- 

a  Normakiation  strategies 
a  Orientation  strategies 
a  Delirium  signs  and  symptoms 

•  Electronic  reports  are  used  to  support  quality  improvement  and  research 

-  POC-specific  content  can  be  extracted  from  the  EHR  for  secondary  use  for  patient 
care,  quality  improvement,  and  research 

-  Reports  are  most  effective  when  used  to  evaluate  end-user  skills  and  to  provide  near- 
real  time  feedback 
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APPENDIX  C.  Dissemination  &  Implementation  of  Evidence-Based  Practice  Frameworks 

Original  Theory: 

Framework  for  Dissemination  of  Evidence-Based  Policy 


*  Build  the  rational* 
for  EBP 

*  Development  of  the 
Community  Guide  and 
other  sources  on 
policy  interventions 

*  Build  source 
credibility 

*  Begin  active 
knowledge  transfer 

*  Identify  target 

.  audience  factors 

*  Identify 
communication 
channels 

*  Uptake  of  the 
program? policy  by  the 
target  audience(s) 

*  Target  adoption  to 
attitudes  and  values  of 
intended  audience 

*  identify  barriers  to 
adopbon 

*  Improve  the  skills  of 
adopters 

*  Provide  training  and 
technical  assistance 

*  Ongoing 
implementation  and 
continued  use  of  the 
innovation 

*  Ensure  policy 
enforcement 

Policy  Outcomes 

Dodson.  Brownson.S  Weiss,  (2012).  Policy  Dissemination  Research,  (p.  440) 


Adaption  of  Model  by  KBN  Research  Team  (in  collaboration  with  TIDIRH  experts): 


Implementation  Theory 
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The  Dodson,  et  al., 
conceptual  framework 
was  adapted  for  this  study 
with  the  addition  of 
assumptions,  conceptual 
details  for  the 
environment,  adoption, 
implementation,  and 
maintenance,  and  two 
intermediary  outcomes 
(knowledge  and  use  of  EB 
practices 


*AiNONI  Hfolth  Qjti* 


u/fciptad  from:  Dooson.  Brownsoo  &  WMt.  {2012} 
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APPENDIX  D.  Baseline  Assessment 


Unit  Descriptions:  Aurora  St.  Luke’s  Medical  Center  (ASLMC) 


Random 

Assignment 

Unit  Type 

#  of  Staff 
(3/2014) 

Associated  Units  # 
Staff  (3/2014) 

Total  Staff  in 

Group 

#  Beds 

Avg  # 
IP/mo 
(3/2014) 

ASLMC  Critical  Care  Units  (5  Units  -  370  Staff;  74=Avg  Staff  /Unit;  SD=16.6  Range=48-92) 

B 

Neurosurgical  ICU 
(3M/1L) 

73 

6KLM  (37) 

10LM  (30) 

140 

16 

96 

B 

Medical/Respiratory  ICU  (8T) 

84 

12S  (25),  12T  (20), 
4KLM  (34),  4EF  (37) 
9LM  (29) 

226 

24 

114 

B 

Surgical  ICU  (3L/3M) 

48 

3CD  (25),  3EF  (40), 
8C  (34),  11S  (24), 
11T  (26) 

197 

14 

70 

A 

Coronary  ICU  (8S) 

73 

5KLM  (20),  10S  (31), 
10T  (32),  11LM  (25) 

181 

24 

118 

A 

CardiovascularSurgical  ICU 
(7T) 

92 

EH 

165 

30 

92 

A 

Clinical  Staffing  Service 
(CSS/Float  Pool) 

35 

35 

ASLMC  Medical/Surgical  Units  18  Units -542  Staff -30  =  Avg  Staff/Unit;  SD=6  Range  =20-40 

11  S 

Orthopedics/Surgical 

24 

SICU 

24 

150 

11T 

Orthopedics/Surgical 

26 

SICU 

24 

154 

12  S 

Oncology 

25 

MRICU 

24 

103 

12T 

Oncology 

20 

MRICU 

24 

94 

3CD 

25 

SICU 

23 

3EF 

Surgical 

40 

SICU 

26 

4EF 

Medical/Telemetry 

37 

MRICU 

26 

143 

4KLM 

Medical 

34 

MRICU 

32 

139 

5KLM 

Medical 

20 

CICU 

28 

130 

6KLM 

Surgical  Neurology 

37 

NEICU 

33 

83 

8  Center 

Med/Surg  Transplant 

34 

SICU 

23 

131 

9LM 

Medical/Telemetry 

29 

MRICU 

23 

10LM 

Medical/Neurology 

30 

NEUICU 

28 

11LM 

Medical/Fleart  Failure 

25 

CICU 

23 

104 

9S 

CardiacSurgical  Step  Dn 

39 

CVICU 

24 

81 

9T 

CardiacSurgical  Step  Dn 

34 

CVICU 

24 

73 

10S 

Cardiac  Procedural 

31 

CICU 

24 

91 

10T 

Cardiac  Medical 

32 

CICU 

24 

121 

Total 

947 

2,568 
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Nurse  Survey 

•  The  Nurse  Survey  was  used  to  evaluate  unit  context  (Alberta  Context  Tool),  perceptions 
about  Research  Utilization,  and  Staff  Nurse  and  Leader  knowledge  of  essential  Practices 
for  all  6  topics  and  workflow 

•  Survey  participant  reported  BSN  (72%)  -  slightly  higher  than  the  general  population 

•  Survey  participants  were  more  likely  to  be  certified  (25%),  more  than  the  general 
population  of  staff  nurses  who  work  on  the  inpatient  units  at  the  facility. 

•  Note:  The  Nurse  Survey  did  not  include  a  question  about  role  to  support  confidentiality. 


Nurse  Survey  -  Participant  Characteristics 


Staff  Nurse  Characteristics 

Baseline  T1  (n=436) 

Populadon  T1  N=947 

«  |  K 

_ 

|  Education 

ADN/Diptoma 

95 

21  % 

BSN 

342 

72% 

550  (63.6%: 

MS  or  Higher 

34 

7% 

»7(1.S%) 

|  Certification  | 

Yes 

t13 

25% 

110/567 

(19.4%) 

|  Shift  Length  | 

12  hour  only 

215 

45% 

8  &  12  hour 

111 

23% 

|  Role  (Advancement  Model}  | 

Competent' Accomplished 

370  (39%)+  212  (22%) 

=  566  (61%) 

Proficient1  Expert 

123  (14%>+  i&9  (21%) 

=  222  (34%) 

Other  (Graduate  Unstaged.  Unknown) 

43(5%) 

RN  Professional  Advancement  Mod 


■ 


Levels: 

•  Newly  Licensed/Competent 

•  Accomplished  (RN  I) 

•  ‘Proficient  (RN  II) 

•  “Expert (RN  III) 


Accountabilities:  “Patient  as  Partner ”  Practice 

•  Clinical  knowledge  and  decision-making 

•  Sharing  knowledge/competencewith  patient'family 

•  Coordination  and  Collaboration 

•  Developing  Relationship  with  Patient/Family 

•  Empowering  the  patient 

•Required  DSN  or  certification  “Requires  BSfl  and  certification 


•  Nurse  Survey  Leader  participants  were  likely  to  have  BSN/MSN  and  certified  when 
compared  with  the  population  of  all  Nurse  Leaders  including  Managers,  Clinical  Nurse 
Specialists,  and  Nurse  Clinicians 


Nurse  Survey  Participant  Characteristics  fiNsih 

»  U  •  N 


Leader  Characteristics 

Baseline  Survey 

Population 

N  |  % 

Manager  (N=24  Units) 

Education 

19 

N=20 

<  BSN 

0 

1  (4%) 

BSN 

13 

68% 

15  (92  5%) 

MS  or  Higher 

6 

32% 

4(17%) 

Certification 

8 

42% 

6(25%) 

Clinical  Nuree  Specialist  /  Nurse  Clinician <N=23) 

Education 

18 

N=23 

ESN 

6 

33% 

8(35%) 

MS  or  Higher 

11 

61 

12(52%) 

Other 

0 

0 

3  (13%) 

Certification 

13 

72% 

13(66%) 

Multiple  Units 

2 

11% 

2 

The  Nurse  Survey  was 
distributed  using  the 
Learning  Connection. 
Funding  was  provided  to 
allow  staff  to  participate 
on  work  time. 

Overall  Participation 
Rate:  48% 

46%  Staff 
86%  Leaders 


0 Aurora  Health  OJf* 
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Unit  Context 


Baseline  (n=4 79) 

N  % 

Completed  KBM  Learning  Modules 

Some  modutes  (1-5) 

173 

36% 

AD  Epic  medu'es  (6-5) 

194 

41% 

None  Of  Dd  not  answer  (0.  missing) 

112 

23% 

Structural  &  Electronic  Resource  Use  at  Work  (last  typical  month)  -  Never/Rarely/NA 

Potees  ano  procedures  {SER  5) 

57 

12% 

CUiccs  Practice  gucetres  (SER  5) 

72 

15% 

Ctimca-  dess«c-5jpt»rt  (CDS)  SER  3 

205 

44% 

Staffing  -Disagree  or  Strongly  Disagree 

Enough  Staff  fof  Neoessa*yWo<rtc 

HI 

30% 

Enough  Staff  to  Deb ver  Quality  Care 

176 

33% 

0Auf*xft  Health  Orff* 


Knowledge  -  Subscale  Scores 


m 


Average 
#  Correct 

Std 

% 

High 

Subscale  Score 

N 

(Mean) 

Dev 

Min 

Max 

Correct 

%  Correct 

Pain  (9Q) 

479 

4.3* 

1.8 

0 

8 

47.3 

89 

Medication  Adherence  (4Q) 

479 

2.6 

1.2 

0 

4 

64.0 

100 

Depression /Suicide  (5Q) 

479 

3.5* 

1.4 

0 

6 

69.2 

100 

Fail  Risk  (6Q) 

479 

3.3* 

1.5 

0 

6 

55.1 

100 

Press  ure  U  leers  (4Q) 

479 

2.0* 

1.2 

0 

4 

49.3 

100 

Delirium  (4Q) 

479 

1.1 

0 

4 

38.7 

100 

Workflow  (9Q) 

479 

5.4* 

2.7 

0 

9 

60.4 

100 

Total  Score  (41Q) 

479 

22.7* 

‘Sgn  tfff  0$ 
Nodtff  by 
Unit  Typ« 

8.5 

0 

38 

55.3 

93 

Key:  Total  Score  includes#  Correct  #  Not  Correct,  and  #  P/lissing  (approx  10%) 
High  %  Correct  refers  to  person  who  scored  the  highest  in  each  group 


Au*ofo  Health  G iff 


Nurse  Survey  Knowledge 
Test  Summary: 

At  baseline,  the  average  total 
score  was  55.3%  correct, 
scoring  lowest  in  their 
knowledge  of  best  practices 
related  to  pain  and  delirium. 
Leaders  scored  statistically 
higher  on  the  Knowledge 
Test  than  Staff  in  Total  and 
for  all  subscales  except 
Medication  Adherence  and 
Delirium.  There  were  no 
significant  differences  in 
Knowledge  Scores  by  Unit 
type. 
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The  KBN  Impact  Study  Observations  Schedule  was  distributed  in  advance.  Units  were  advised  to 
conduct  care  under  usual  patient  care  circumstances  with  no  requirement  for  leader  presence. 

Blinded  and  trained  KBN  Team  members  reviewed  the  prepared  patient  list  and  identified  the  nurse 
who  had  the  highest  number  of  selected  patients  to  observe  for  the  6  hour  session.  The  observers 
focused  on  their  assigned  nurse  during  patient  care  and  documented  their  activities  on  the  data 
collection  tool  including  when  they  participated  in  daily  unit-based  Outcome  Facilitation  Rounds  (OFTs). 
Observers  also  recorded  instances  of  Nurse  Leader  interaction  on  the  unit  regarding  the  study 
phenomenon  (e.g.  during  huddles,  rounds,  or  on  the  unit  with  the  staff. 


Observations  by  Unit 


Details 

Baseline 

Population 

N 

% 

Avg  IP/mo 

Total  Observations 

379 

100 

100 

Critical  Care  {5  units) 

85 

22.4 

159 

Mcde'ate  Acuity  (5  un.ts) 

69 

18.2 

162 

Blended  Acuity  (2  units) 

37 

9.8 

140 

]  Combined  Med-Surg  (6) 

102 

26.9 

172 

Ved>cal  (5  units) 

36 

22.7 

152  1 

Day  ShiftObservations 

341 

90% 

Average  /  Ratios 

Vear 

SO 

Range 

*  Obs  Pts  i  Unit 

16.5 

2.3 

11  -21 

I  RN  :  Ft  Rato-  ICU 

1.9 

0.4 

1  -2 

1  RN:Pt  Ratio  Non-ICU 

1  ■  —  -  — 

4.0 

0.7 

2-6  | 

Patient 

Baseline  T1 

Characteristics 

(N=379) 

N 

X 

X-SD 

Range 

Age  (yrs) 

64.2+10.6 

16-101 

Day  of  Stay 

8.0  +  12.2 
1-128 

Male 

179 

47.3 

Surgical 

'£4 

40.6 

30  Day 

Readmit 

103 

27.2 

Isolation 

83 

21.9 

Hx  Dementia 

35 

9.2 

Not  Taking 

Meds 

As  Prescribed 

11 

2.9 

Observed  RN  Characteristics  -  All  Uni 


Cnaractertitic* 

BasoUrw 

n 

Popuiacon  (U=M 7  RNs) 

N 

% 

UnitRN  Observed 

110 

12.2 

947 

Patients  Observed 

All  Units 

379 

Patients  Observed  by  RN  Type 

UnitRN 

368 

97 

Float  RN 

11 

3 

Patients  Observed  by  RN  Job 

Competent 

174 

45.9 

39% 

Accompisned 

69 

182 

22% 

P'of'oent 

42 

11.1 

13% 

Expert 

01 

10.1 

21% 

Other 

33 

8.7 

5% 

The  Observers  recorded 
any  instances  when  they 
observed  the  nurses 
speaking  about  or  doing 
behaviors  that  related  to 
any  of  the  6  evidence- 
based  topics  under 
study.  The  Observers 
were  blinded,  meaning 
that  they  were  observing 
without  knowing  specific 
history  or  care  details 
about  the  patient. 

Each  unit  had  an 
average  of  16.5  patients 
who  were  observed 
during  the  baseline  time 
frame. 

Most  of  the  observations 
occurred  on  the  Day  Shift 
(90%),  with  unit  Staff 
Nurses  (97%).  More 
than  half  (61  %)  of  the 
observations  were 
completed  with  RNs  in 
competent  or 
accomplished  RN  Job 
code. 

When  the  observations 
were  finished  the  sheets 
were  reviewed  and 
entered  into 
SurveyMonkey  as  the 
first  step  tin  the  audit 
process. 
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Nurse  Leader  Observations  -  Baseline 


mm 


Nurse  Leaders  were  observed  for  Knowledge  &  Use  of  EBP  Behaviors 

■  2  observation  days  per  non-critical  care  unit  (63%) 

■  3-5  observation  days  for  each  critical  care  unit  (37%) 

■  5  evening  shifts  did  not  yield  any  leader  observations  (excluded) 

■  PCMs  (n=23)  -  49  Observation  Days 

■  CNSs  (n=15)  -  39  Observation  Days 

•  NCs  (n=6)  - 10  Observation  Days 

•  Locations:  Huddles,  OFTs  (Rounds)  or  on  Unit  interacting  with  Staff 

•  Outcome  Facilitation  Team  Meetings  (Rounds): 

-  Mean  duration=41'+  14' 

-  Leaders  present  74% 

■  Observation  Frequency  varied  by  Unit  Type 

Critical  Care:  44%;  Comb  Med/Surg:  33%.  Blended:  13% 

Medical:  5%;  Moderate  Acuity:  4% 

■  Behavior  use  varied  by  Role;  Leaders  demonstrated  more  than  one 
behavior  when  they  were  observed. 


Aurora  Health  Core  ■ 


Nurse  Leader  Observations  Baseline 


Huddles 


OFTs 


On  Unit 


Pain 


MedAdh 


Suic 


Falls 


PU 


Deliriun 


Actual  Otis 


4'49  :s%) 


1  1 

5  8  4 

6  4  3 

4  8 

18/39  (48%) _ 


2/10(20%) 


Nurse  Leaders  were 
observed  during  unit 
observations  during  Q2 
2014  (3  months). 

Observers  recorded  any 
instances  when  they 
observed  the  leaders 
speaking  or  doing 
behaviors  that  related  to 
any  of  the  6  evidence- 
based  topics  under 
study. 


Note:  33%  of  Leader  Observations  occurred  during  OFTs 

Summary-  Nurse  Leader  Observations:  At  baseline,  nurse  leaders  were  observed  engaging  with 
staff  primarily  during  daily  “OFT”  rounds  (93%). 


CNSs  had  the  majority  (67/75;  89%)  of  documented  observations  where  they  were  seen 
demonstrating  EB  knowledge  and  use  of  supportive  leader  behaviors,  Some  of  the  CNSs  in  Study 
Group  B  were  already  using  the  electronic  reports.  Pain,  Falls,  and  Pressure  Ulcers  were  the  topics 
where  Leaders  were  most  often  observed  engaging  staff  about  evidence-based  practice. 


39 


Award  #:  W81XWH-1 3-1 -0034 
Aurora  Health  Care  -  Year  3  Report 


Population  Characteristics  (18  Units)  - 


Population 
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Patient  Survey  (N=1 84) 


■  Demographics- Med/Surg  Patients 

-  Age:  Average^  61.2  yrs  (SD=15.8:  20-99  yrs) 

-  Female  (52%) 

-  Race/Ethnicity 

•  Caucasian  (73%) 

•  White'  Hispanic  (4%) 

•  5  lack  /  Afro-Amertcan  (20%) 

•  Asian  (3%) 

-  30  Day  Readmissions(27%) 

-  Learning.Assessments 

•  Missing  (7%) 

•  Current  Stay  (50%) 

•  Prior  Stay  (43%) 

-  No  Barriers  to  Learning  (87%) 


Patient  Perceptions  about  Teaching 


Baseline 

Subjects  vrho  Recall  RN  Teaching 

N=166/184  (90%) 

Metnod  (how)  Ejoararw*  aurrg  Care 

153 

91% 

Eoaratcrs  during  Ttachhg  Sesscr 

4 

2% 

Cerrcrstratw 

3 

2% 

3ooo  Way  to  Lear n  =  Ye* 

153 

92% 

ran*  ot  Day  sospecaem 

43 

29% 

Any  tne  oo4ng  care 

79 

48% 

nme  of  Day:  Acceceoie-Yes 

152 

92% 

Nurses  Explain*  Thing*  Wen 

148 

89% 

Neported  Asung  Quest  ons 

138 

83% 

Nurse*  Listen 

153 

92% 

Nurse*  Kind  ana  Courteous 

160 

96% 

Nurses  involve  You  m  YOur  Plan  of  Care 

98 

59% 

Patient  Survey: 

Med/Surg  patients 
(N=581)  were  screened 
and  recruited  to 
participate  in  the  Patient 
Survey.  Patients  were 
asked  about  the 
teaching  method  and 
the  qualities  of  the 
nurse  patient  interaction 
and  about  their 
preferences.  They  were 
also  asked  if  they 
recalled  specific  details 
about  to  determine  if  the 
nurses  to  delivered  the 
evidence-based 
information  to  patients 
and  to  what  extent  that 
the  patient  report 
knowledge  and  use  of 
the  recommended  care 
practices. 
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APPENDIX  E.  Study  Intervention  with  Varied  Maintenance  Strategies  (Group  A  &  B) 


KBN  Impact  Study  Consort  Diagram^^ 
Clustered  Randomized  Controlled 


Baseline  KBN  Deployment  Evaluation 


Critical  Care i 


M=5)&  Medical/Surgical  (N=1 8)  Units- 100%  at  Sit 
Observations/Audits.  Patient  Survey  &  Nurse  Sur 


Implementation 


-i- 


Post-lntervention 

Assessment 


Post-Intervention 

Assessment 


C5 


Aurora  Health  Care • 


Baseline  findings  indicated 
that  knowledge  scores 
were  low  in  basic  evidence- 
based  practices  and  nurse 
leaders  had  limited 
interaction  with  staff  outside 
ofOFTs.  Behavioral 
Expectations  were  drafted 
and  distributed  during  the 
training  and  to  guide  units 
in  selecting  their  priority 
projects  for  implementation. 

Additional  funding  was 
provided  to  engage  (2)  unit 
preceptors  for  each  unit  to 
support  unit  implementation 
efforts. 


Training  Sessions:  Audit/Feedback  Results  with  Behavioral  Expectations  and  EB  Content 


Knowledge  Based  Nursing  (KBN)  Behavioral  Expectations 

Goal:  Promote  use  of  evidence-based,  oat  ient<e  rite  red  practices  with  accurate,  meanineful.  and  efficient  documentation 


Navigator  /  Flowsheets /  Care  Planning 

•  Use  Navigators  to  ensure  timely  completion  of  all  components  of  the  workflow 

•  Use  "Monitoring"  flowsheets  (e.g.  Frequent,  PCA/Epidural, Complex, Cardiac,  Neuro,  Stroke) -ON ESTOP  SHOP 

•  Review  “Active  LDAs”  on  admission/transfer  (edit  properties/resolve) 

•  Manage  "Best  Practice  Advisories"  (don't  acknowledge  if  not  done) 

•  Create  and  maintain  PLAN  OF  CARE  (select/remove/edit;  Review  individualized  plan  early  in  shift) 

•  Know  howto  see  “Care  Plan  &  Patient  Education*  (LCVon  Patient  Story  or  All  on  Care  Plan  Overview) 

•  Ensure  care  plan  outcome  documentation  accurately  reflects  progress 

•  Use  "Care  Plan"  notesto  capture  issues  and  interventions  when  outcomes/interventions  NOT  MET 


Mental  Status  (Delirium  Risk/ Actual)/  Depressive  Symptoms 

•  Ask  questions  to  evaluate  mental  status  (see  ref  text) 

•  Use  parameters  beyond  orientation/memory  to  describe  confusion  (LOC,  attention,  &  thought  process) 

•  Effectively  identify  change  from  baseline  OR  fluctuating  status 

•  Use  CAM-ICU  for  ICU  patients  who  are  unable  to  speak  (e.g.  intubated  patients) 

•  Select  and  carry  out  delirium  prevention  activities 

•  Report/confirm  delirium,  aggressively  treat  cause(s)  and  manage  withnon-pharmacological  interventions 
Search/treat  additional  causes  when  symptoms  persist  (>  24  hrs) 

•  Limit  use  of  anti-anxiety  and/or  sedation  medication 

•  Use  depression  screening  tools  to  assess  and  support  medical  (BPA)  and  symptomatic  (manual)  patients 


Pain  -  Comfort/Function 

•  Select  ONE  appropriate  pain  assessment  tool  (self-report  vs.  behavior;  age-appropriate) 

•  Partner  with  patient  to  set  realistic  comfort  function  goal  (Typical  =  3  or  4) 

•  Evaluate  "acceptable"  paincontrol  based  on  reported  pain  rating  at  or  belowtheirgoal 

•  Identify  symptoms  of  "opioid  tolerance"  and  dealswith  heightened  pain  symptoms  at  end  of  drug  duration 

•  Individualize  Pain  Care  Plan:  goals  based  on  tool/department,  pain  interventions  (vs.  comfort) 

•  Monitor  function  with  actual  observations  usingtool  descriptors:  Braden,  Morse,  ADL  Index 

•  Conduct  "Get  Up"  (Egress)  testing  when  needed:  Bed  mobility,  dangle/stand,  walk 

•  Support  early  mobilization:  upright  w/in  24  hrs  (if  tolerated);  Up  2-4x/day  (increase 
frequency/intensity/duration);  in  hallway  l-2x/day 

•  Monitor  pain  management/oversedation  using  POA/  Epidural  /Opioid  flowsheet 


Patient  Education 

•  Identify  barriers  and  preferences  for  learning  durir^  admission  (e.g.  literacy  question,  mental  status/affect, 

•  Screen  older  adults  (65+yrs)fbr  cognitive  impairment:  Orientation,  Memory,  Cognition  Test  (OMCT) 

•  Identify  learning  needs  -  many  patients  have  had  with  prior  stays 

•  Deliver  (and  document)  individualized  education  withcomments about  contentor  how  barrieis addressed 

•  Use  visual  aids  to  review  important  messages  about  what  patients  need  to  DO  ("patient  engagement") 

•  Avoid  redundancy:  If  patient  verbal  ized  understanding- why  repeat? 

•  Conduct  "focused  medication  assessment"  to  identify  issues  (especially  if  readmission) 

•  Implement  interventions  if  at  risk  or  actual  medication  nonadherence  issues 


Optimization  Training _ Fal1 ; 

KNOWLEDGE  BASED  NURSING  (KBN) 

Infusing  evidence-based  practice  (EBP)  recommendations  into  nursing  workflow  using 
electronic  health  record  content,  policy,  and  clinical  decision  support  (CDS). 


KBN  Core  Components 


m 


in 


Aim:  improve  nurse/nurse  leader  knowledge  and  skills  in  the 

•  Delirium  •  Depressive  Symptoms  •  Falls  •  Medication  Adherence  •  Pain  •  Pressure  Ulcers 


Self-evaluation/Reflection 


Identify  priority  issues: 


AGENDA 


Identify  strategies  to  reduce  barriers: 


impact  Study 
Basefire  Findings 


Breakout 

Sessions 


{4)40  minute 
sessions 


What  personal  behaviors  you  will  consider  changing  as  a  result  of  this  session? 


C3 
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ces  impact  nursing-sensitive  patien 


_ 
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Training  Session  Evaluation  Results 


Sessions:  44 

Attendance:  Leaders  =45  and  RN  Staff  =  849 
Response  Rages:  Leaders  (93%)  and  RN  Staff  (94%) 


Leader  N=42,  RN  N=801 

1 

At  the  end  of  this  program,  1  am  able  to:  Briefly  describe  what  Knowledge-based  Nursing 
(KBN)  is  and  how  evidence-based  practices  are  embedded  into  the  electronic  health 
record  to  support  patient  care 

Leader 

RN 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

39 

92.86% 

778 

97.13% 

2 

At  the  end  of  this  program,  1  am  able  to:  Report  two  key  findings  from  baseline  that  you 
find  most  relevant  and  appropriate  for  considering  as  a  practice  change 

Leader 

RN 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

39 

92.86% 

741 

92.51% 

3 

3a 

3b 

3c 

3d 

At  the  end  of  this  program,  1  am  able  to:  Validate  and/or  update  your  knowledge  and 
skills  in  performing  and  efficiently  documenting  essential  evidence-based  practice 
recommendations  related  to: 

Leader 

RN 

Navigators/Flowsheets/Care  Planning 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

39 

92.86% 

776 

96.88% 

Mental  Status 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

42 

100% 

786 

98.13% 

Pain 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

41 

97.62% 

784 

97.88% 

Pt.  Education 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

41 

97.62% 

723 

90.26% 

5 

How  committed  are  you  to  implementing  these  practices? 

Leader 

RN 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

40 

95.24% 

765 

95.51% 

6 

Would  you  recomment  this  training  session  to  others  in  your  role? 

Leader 

RN 

N 

% 

N 

% 

Strongly  Agree  +  Agree 

39 

92.86% 

687 

85.77% 

7 

Identify  breakout  sessions  where  your  knowledge  of  the  topic  increased  after 
participation: 

Mental 

Status/ 

Depression 

Pain/Functio 

n 

Patient 

Ed/Med 

Adherence 

Navigators/F 
lowsheets/C 
are  plan 

Leader  (N) 

26 

16 

13 

28 

Leader  (%) 

61.90% 

38.10% 

30.95% 

66.67% 

RN  (N) 

492 

264 

220 

489 

RN  (%) 

61.42% 

32.96% 

27.47% 

61.05% 
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Unit  Implementation 

Units  were  randomized  into  two  groups  with  varied  maintenance  strategies 

All  units  received  unit-level  results  and  asked  to  prioritize  implementation  efforts.  Leaders  and 

preceptors  attended  a  formal  2  hour  training  course 

•  Group  A:  Unit  feedback,  Leader  and  Preceptor  Course  with  directions  to  use  usual 
practices  to  monitor  and  support  implementation 

•  Group  B:  Unit  feedback,  Leader  and  Preceptor  Course  and  additional  training  (below) 
re:  the  use  of  the  electronic  “Key  Performance  Indicator”  (KPI)  Unit  Details  Report  to 
support  them  to  monitor  the  use  of  the  practices  with  limited  time. 
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Training  for  Varied  Maintenance  Strategy  for  Group  B:  Additional  training  and  support  re:  the 
“Key  Performance  Indicator”  Unit  Details  Daily  Report  to  support  monitoring 
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Preceptor  Tracking  Tool 

Name: _ 

Unit: -  Aurora  Health  Care' 


[DATE _ #  OF  HOURS  ACTIVITY  (check  all  that  apply)  Comments 


Preceptor  Training  -  Parkway  Training  Center 

z  Education  □  Leader  collab.  □  Auditing  □  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Education  □  Leader  collab.  z  Auditing  z  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Education  □  Leader  collab.  z  Auditing  z  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Formal  f/u  1ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Education  z  Leader  collab.  z  Auditing  z  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Education  z  Leader  collab.  z  Auditing  z  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

z  Education  z  Leader  collab.  z  Auditing  z  Informal  f/u  w/staff  z  Preceptor-Preceptor  collab. 

|z  Formal  f/u  (ACC  meeting)  z  OFT's/Huddle  z  Other: 

1 

Total  Hours 


Hours  should  be  charged  to  the  KBN  Study  org  and  area  (505-3969).  Either  use  the  Kronos  clock  or  work  with  your  timekeeper  to  transfer  hours. 
Please  return  this  form  to  Nichole  Nikolic  (email:  nichole.nikolic^aurora.or&or  fax:  414-219-5693) 
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Additional  Information  Gathered  from  Preceptors 


Preceptor  Characteristics 


KNOWLEDG6-BAS 

M  U  R  S  I  N 


N 

% 

M 

SO 

Range 

Median 

Formal  Training 

RN  Years  Total 

48 

100 

64 

6.7 

0  5-34 

5.0 

Critical  Care 

10 

21 

10.7 

10.2 

1.0-34 

8.0 

Mod  Acuity 

10 

21 

4.6 

5.0 

0.5-11 

50 

Blended  Acuity 

4 

8 

11.8 

10.6 

3.0-27 

8.5 

Combined  Med/Surg^ 

12 

25 

4.5 

3.1 

1.0-10.0 

4.5 

Medical 

10 

21 

3.9 

2.3 

0.6-7 

3.5 

Preceptor  Years  Total 

48 

100 

44 

5.4 

0-25 

3.0 

17  (35%) 

Critical  Care 

10 

21 

7.0 

7.4 

0-24 

4.0 

4  (40%) 

Mod  Acuity 

10 

21 

2.8 

3.5 

0-10 

1.5 

1  (10%) 

Blended  Acuity 

4 

8 

10.5 

10.1 

2-25 

7.5 

2(50%) 

Combined  Med/Suro 

12 

25 

2.9 

2.7 

0-8.0 

3.0 

5  (42%) 

Medical 

10 

21 

2.3 

2.3 

0-5 

1.0 

5  (50%) 

Study  Hours  /  Preceptor  Total 

1007 

100 

21.0 

130 

0-473 

20.9 

Avg  Hrs  /  Unit 

Critical  Care 

267.5 

26.6 

26.75 

16.2 

6.5-47.3 

24.9 

53.5±28.4 

Mod  Acuity 

237.5 

23.6 

23.75 

9.3 

9.5-38.8 

23.25 

47.5  i  18.5 

Blended  Acuity 

38.3 

3.9 

9.7 

8.1 

1.8-17.5 

9.8 

19.4+19.6 

Combined  Med/Surg 

263.3 

26.1 

21.9 

9.9 

0-50.75 

24.3 

43.9±21.5 

Medical 

184.5 

18.4 

185 

13.3 

4.8-29.3 

18.1 

61.5  +  21  7 

Float  Pool  (15  RM-10  Preceptor  Yrs;  NoTra*r) 

15.5 

1.5 

7.5 

0,7 

7.3-83 

7.8 

15.5 

Preceptor  Characteristic* 

5  GSfiK 

KNOWLEDGE-QASEC 

NURSING 

The  Preceptors 
selected  to  support  the 
project  were  varied. 
Several  were  highly 
seasoned,  but  many 
were  not  formally 
trained  with  less  than  3 
years  of  experience. 

Preceptors  at  the 
primary  site  utilized 
their  grant  funded 
hours  (Average  ranged 
from  9.8  to  25  hrs/per 
person). 

: 

N 

% 

M 

so 

Range 

Median 

Training 

RN  Years  Total 

48 

100 

6.4 

6.7 

05-34 

5.0 

A  Surveillance  Method  (Usual) 

18 

37.5 

7.8 

9.0 

0.5-34 

5.0 

B  Surveillance  Method  (eReDorts) 

30 

62.5 

5.7 

5.1 

1.0-27 

5.0 

Preceptor  Years  Total 

48 

100 

4.4 

5.4 

0-25 

3.0 

17(35%) 

A  Surveillance  Method 

18 

38 

4.7 

6.4 

0-24 

2.0 

6  (33%) 

B  Surveillance  Method 

30 

62.5 

4.2 

4.9 

0-25 

3.0 

11(37%) 

Study  Hours/Preceptor  Total 

1007 

100 

21.0 

13.0 

0-47.3 

20.9 

Avg  Hrs /Unit 

A  Surveillance  Method 

347.5 

34.5 

19.3 

10.1 

6.5-38.8 

19.0 

38.6+19.8 

B  Surveillance  Method 

659.5 

65.5 

22.0 

14.5 

0-47.3 

24.1 

44.0+  24.7 

Preceptor  Efforts 

Surveillance  Tool  Skills 

30 

% 

Logs=Audits  (unit) 

N 

% 

A  Surveillance  Method 

n/a 

n/a 

A  Surveillance  Method 

8 

89 

B  Surveillance  Method 

14 

50 

B  Surveillance  Method 

13 

87 

Logs=Leader  Collaboration 

24 

Logs=Education  (unit) 

24 

A  Surveillance  Method 

4 

44.4 

A  Surveillance  Method 

6 

67 

B  Surveillance  Method 

9 

90.0 

B  Surveillance  Method 

11 

73 

Logs=lnformal  F/up  (unit) 

24 

Logs=Formal  F/u  (unit) 

A  Surveillance  Method 

6 

67 

A  Surveillance  Method 

1 

11 

B  Surveillance  Method 

12 

80 

B  Surveillance  Method 

5 

33 
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APPENDIX  F:  Findings  from  Primary  Study  Site 


Unit  Descriptors:  Total  and  by  Unit  Type 


Icm  l'J] 


Cnaractanstci 

Basavi*  (T1 ) 

Population  fri) 

Post  (T2) 

Population  (T2) 

r»sr  1 

N 

X 

Average  /  Unit 

N 

X 

average  fUnli 

(W 

N  ureas 

912 

396  ^  206  urn 
R  20-92 

968 

432  +23  4^  urt 
R  X  -  102 

0.66 

BSN 

650 

71 

283  +151 

R  U  -63 

727 

75 

31  6  + 15.7 

R  15  -  77 

0.17 

C+rtf+d  E;gt>« 

110*567 

19  4 

48+43 

RO-14 

99556 

178 

43+42 

R  0-20 

a7i 

Ron  =  Competent 

370 

40-6 

161  +73 

R  8-31 

374 

386 

16  3  +62 

R  6“  31 

093 

Ron  =  Export 

199 

21  S 

87  +97 

R  0-26 

224 

231 

97  +  108 

R  0-45 

0.72 

Ttimovsr 

13^  ±y 

R  5-2SN 

Tot*  Moure  per  Pt  Day 

23^rxs 

100 

1175  +3  5  T 

23 

IX 

1211  +3.7  T 

074 

RN  Hours  per  Patent  Day 

23  ur.ts 

IX 

80  +  42  RN 

23 

IX 

82+44  RN 

oss 

Critical  Care  QStinas) 

RN  74.  .rt 

155+12 

RN  77\rt 

163  + 1  4 

0.42 

Vooerate  Acusy  (5  ufts; 

R\U\rt 

61  +06 

RN36 

61+02 

013 

5«-oeo  Acut,  ’2  „‘ts: 

RN  29U* 

61  ±09 

RN  31  \ri l 

86  ±09 

061 

Can  i/ecSug  ;«  unis 

RN  30>rt 

57+02 

RN  33.  *.r.a 

56+02 

0.61 

Vecncai  ;5  unis: 

RN  27,'-Tt 

57+03 

58+04 

0.61 

w Aurora  Health  C off* 


The  Post-Intervention 
Assessment  was 
conducted  during  Q2  of 
2015  (6  months  after  the 
Audit/Feedback  and 
Training  Sessions). 

The  Units  were  evaluated 
against  the  baseline 
findings  1  year  prior  (Q2, 
2014). 

The  Units  were  the  same 
(not  statistically  different) 
when  the  post  evaluation 
was  compared  to  baseline 
on  key  descriptors 
including  number  of 
nurses,  roles,  and  Hours 
per  Patient  Day  (HHPD). 


Observations  by  Unit 

i 

i 

The  Observations  and 

Audits  were  conducted 
per  protocol  during  the 
post-intervention  period. 

Note:  The  Observations 
were  scheduled  during  the 
day  shift  exclusively 
during  the  post-time 
period  to  improve  data 
capture  of  key  elements 
including  activities  during 
OFT  Rounds. 

■  Characteristics 

Baseline 

Population 

Post-Intervention 

Population 

N 

y. 

Avg  IP/mo 

N 

y. 

Avg  IP/mo 

Total  Observations 

37S 

IDO 

ISO 

361 

100 

155 

Critical  Care  (5  units) 

85 

22.4 

155 

71 

197 

152 

Mcd+'ate  Acuity  (5  units) 

09 

18.2 

102 

73 

20.2 

166 

Blended  Acuity  (2  units) 

37 

9.8 

140 

38 

10.5 

158 

Combined  MediSurg  (6) 

102 

28.9 

172 

99 

27.4 

104 

Medical  (5  units) 

36 

22.7 

152 

80 

22.1 

ieo 

V.  SC 

Range 

M.  SD 

Range 

#  Obs  Pts  /  Unit  (Average) 

16.5+2.3 

11-21 

15.7+1.9 

13-19 

RN:Pt  Ratio- ICU(Avg) 

1.5+0. 4 

1  -2 

18+0.5 

1  -2 

RM:Pt  Ratio  Non-ICU  (Avg) 

4. 0+0. 7 

2-3 

4.2+0.7 

3-5 

Day  ShiftObservations 

341 

SC’--. 

361 

100% 

_ 
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Nurse  Leader  Observations: 


Nurse  Leader  Observations  -  Post 


Nurse  Leaders  were  observed  for  Knowledge  &  Use  of  EBP  Behaviors 

■  59  Observation  Days 

19  Critical  Care;  12  Mod  Acuity;  4  Blended  Acuity 

14  Combined  Med/Surg;  10  Medical 

All  ‘Day*  Shift  observations  (increase  in  leader  observations) 

PCMs  -  59  Observation  Days 

CNSs  (15)  -45  Observations  Days  on  CNS  Staffed/Covered  Units 
NCs  (6)  -  14  Days  on  NC  Staffed  Units 
Locations:  Huddles,  OFTs  (Rounds),  and  on  Unit 
Day  of  Week;  Avg  20%  per  day  -  Range:  15%  Tues  -  24%  Wed 
Huddles:  15/60  -  90%  <10”;  75%  Leader  present 
OFTs:  Duration  Mean  =38’+13:  88%  Leader  present 


Nurse  Leader  Observations  Post 


I 


m\ 


m 

MU* 


_ All  Units  (23  Units;  N=59  Observation  Days 

PCM  (N=23)  CHS  (n=1 5) 


HC  <n=6 


£  S>  8 

©  © 

■*-  03 

8 

CD  ® 

a> 

2  U  o 

CO  *o 

CD 

CD  "O 

fill  1  5 

O  2  2  O  CD 

Is 

o  2  ° 

a>  j- 

©  o 

.c  -o 

O  CD 

■5  ®  o> 

■e  "  §  ® 

m  o  °  o  o 

Q.  E  c  CO  = 

S  E  3 

CD  = 

S  o-  E  S  « 

0 >  ®  Jt  o  o 

OJ  ®  ® 

o  o 

lO  ®  ®  ““  O 

H  □  in  O  O 

D  o:  Q  00 

o  o 

O  tr  Q  00  o  < 

OFTs 

1  6  13  7 

0  13  25  11  1 

1  0  7  4  2 

|  On  Unit 

3  | 

1  1  1 

l  3 

1  Pain 

2  5  1  1 

2 

i  4  e 

3  e  2  1 

i 

5  1 

6  5  5 

3  6  3 

4 

2  1 

1 

1 

3  1 

2  1 

2  11 

I  MedAdh 

Falls 

PU 

4  3 

Delirium 

3 

Total 

Instances 

1  10  14  7 

0  13  26  11 

0  7  4  2 

15'59{25%> 

20/45144%) 

7/14(50%) 

Leader  EB  Behavior  T1  vs.  T2 


Adjusting  to  all  Day  Shift 
observations  increased 
the  frequency  of  Leader 
observations. 

Leader/staff  interaction 
continued  to  be  primarily 
observed  during  the  daily 
OFT  meetings  (91%). 
Leaders  increased  the 
instances  when  they  were 
observed  to  demonstrate 
knowledge  and/or  coach 
on  the  focused  topics 
during  the  post¬ 
intervention  time  frame. 

Only  one  instance  of 
electronic  report  usage 
was  observed. 
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Nurse  Survey  Results: 


RN  Survey  Participant  Characteristics 


Nurse  Survey 

_  < 
% 

5ample/Population 

Details 

lllp 

■ 

Staff  Nurse 
Characteristics 

Baseline 

(n=d3S) 

Ft  Population 
N=947 

Post-In  t erven  n  on 
(n=437) 

72  Population 
N=9S  8 

■ 

N  X 

N  X 

|  Education 

ADN/Dipkxna 

S3 

21  % 

85 

‘3% 

BSN 

342 

72% 

350  (38.9%) 

339 

73% 

727  (75%) 

MS  or  Higher 

34 

17  (1.6%) 

43 

8% 

19(2%) 

Certification 

Yes 

113 

25% 

110537 

(134%) 

106 

24% 

99(550 
(17  85.) 

1  Shift  Length 

12  hour  only 

215 

45% 

215 

465, 

8  &  12  hour 

111 

23% 

122 

265, 

|  Role  (Advancement  Model) 

Ccnpaert 

370  (39%} 

374  :39%; 

Acecnwie-ec 

212  :22V 

195  i20%; 

ProOc»* 

123  . 14%  J 

123  ;13%; 

Boed 

199  (21%) 

224  03%) 

crer 

43  :s%> 

52  $%) 

The  Nurse  Survey  was 
completed  after  the 
Observations  and  Patient 
Surveys  were  completed. 
The  Post  Intervention 
sample  contained  182 
subjects  that  had 
participated  at  baseline. 
In-depth  analysis  was 
conducted  to  evaluate  the 
matched  sample  and 
compare  it  to  the  entire 
sample. 

It  was  noted  that  the 
subjects  who  scored  the 
highest  on  the  baseline 
test  did  not  repeat  and 
many  of  the  “new” 
participants  did  not  attend 
the  training. 

Participants  were  still 
more  likely  to  be  BSN  and 
certified  when  compared 
with  the  sample  of  all 
nurses  on  the  unit. 
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iHjl 


fm* 

m  u  n  «  «  h  « 


Unit  Context 


Characteristics 

Baseline 

(n=479) 

Post-In  rerven  n  on 
(n= 467) 

N 

% 

N 

% 

Completed  KBN  Learning  Modules 

Some  mcdues  (1-5) 

173 

36 

205 

44 

A*  Epic  modules  (6-8) 

194 

41 

161 

34.5 

None  or  Dd  not  ahs*er  (0/mtss  ng| 

112 

23 

101 

21.5 

Structural  &  Electronic  Resource  Use  at  Work*  (lasttypical  month)  -  Never/Rarely/NA 

Po'ices  ana  procedures  ;SER  5) 

57 

12 

38 

1.6 

Clin  tea  Practice  gu oekres  (S£R  9) 

72 

15 

74 

16.2 

CfiTHca-  aeoson-supocft  (CDS)  SER  3 

205 

44 

195 

42.8 

Staffing  -  Disagree  or  Strongly  Disagree 

Enough  Staff  fc»  Necessary  Wortc 

141 

30 

211 

46 

Enough  Staff  to  DefuerC-uaJrty  Care 

176 

38 

243 

54 

•Resources  Never  to  'are  useof  ^cary  (77%).  textoooks  (78%)  or  .oumais  (58%) 


Unit  Context  -T1  vs.  T2 


All  Units  and  Participants 


5.0 


Research  Utilization  T1  v.  T2 


Ail  Units  and  Participants 


The  Context  evaluation 
was  reviewed  -  fewer 
post-survey  participants 
had  completed  the 
training  modules. 

More  reported  using 
policy  and  procedures  in 
their  daily  work. 

More  (statistically 
significant)  reported  that 
staffing  was  not  enough 
for  work  or  to  deliver 
quality  care. 

The  Alberta  Context 
Evaluation  showed  that 
the  mean  scores  across 
most  subscales  continued 
to  be  rated  above  the 
midpoint. 

The  “Resource”  and 
“Slack”  variables  however 
were  below  the  midpoint 
with  a  drop  in  perceptions 
about  Staffing  -  a 
subscale  that  was  rated 
statistically  lower  for 
Leaders  and  Staff. 


Perceptions  about 
Research  Utilization 
remained  high,  although 
participants  reported  a 
statistically  significant 
reduction  in  the 
organizational  supports 
for  research  between 
baseline  and  the  post 
period. 
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Knowledge  Test  Results  -  Comparing  Difference  Scores 


oro  Health  i 


t  Oi 0K. 


Knowledge  Test  scores 
improved  significantly  for 
Pain,  Delirium,  Workflow, 
and  Total  Score.  The 
Scores  went  down  re: 
Risk  for  Falls. 


The  Patient  Volumes  and 
Staffing  Hours  were 
unchanged,  but  the  LOS 
for  the  time  period  when 
down  significant  by  almost 
one  day. 


Outcome:  T1  (Q22014)  vs.  T2  <Q2  2on 

j)  G3BK! 

"  KNOWLf tfGE-BASCO  i 

N  U  P  S  1  N  6  i 

1  Metric  (3  month  average) 

Baseline  (Q2,  2014) 

Post-Intervention  (Q2,  2015) 

Ttest 

N 

M 

SD 

Range 

N 

M 

SD 

Range 

(P) 

Average  Length  of  Stay 

23 

468 

1.1 

3  0-68 

23 

3  86 

0.6 

2.6  -5.0 

0.003 

Admissions/Month 

23 

108.7 

26.7 

56-160 

23 

99.3 

22.0 

46-130 

0  20 

30  Day  Readmission  Rate 

23 

0.17 

0.06 

0.09-031 

23 

0  18 

0.07 

0.06-0.38 

055 

Press  Ganey  Pain  Management 

23 

0.83 

0.1 

0.69  -1.0 

23 

0.83 

0.1 

0.64-1.0 

0.88 

Press  Ganey  Patient  Education 

23 

0  90 

0.1 

0.72  -1.0 

23 

0  89 

0.1 

0.77-1.0 

0.65 

NDNQI  Fall  Rate  (#/1 ,000  PD) 

23 

2.04 

2.0 

0-6.5 

23 

204 

1.3 

0-4.0 

0.99 

NIDNQI  Injury  Fall  Rate  (#/1000  PD) 

23 

0.60 

0.8 

0-2  6 

23 

0.42 

0.4 

0-1.04 

0.34 

NDNQf  Pressure  Ulcers  (#/1000  PD) 

23 

4.65 

6.3 

0-28 

23 

2  04 

3.0 

0-9.5 

0.08 

NIDNQf  Stage  II  PU  (#/1 000  PD) 

23 

4  29 

6.4 

0-28 

23 

2.03 

3.0 

0-9.5 

0.14 

KPI:  Integumentary  Assmnt  w/in  4  hrs 

23 

093 

02 

0-100 

23 

0.98 

001 

0  96  -1.0 

0  22 

KPI:  Braden  Assessment  w/in  4  hrs 

23 

0.88 

0.2 

0-0.99 

23 

0.93 

0.04 

0.86  -  1.0 

0.21 

KPI:  Morse  Risk  Assessment  w/in  8hrs 

23 

091 

0.2 

0-  10 

23 

0  98 

0.02 

09-1.0 

0.17 

KPI:  Not  Taking  Meds  Rate 

23 

0.04 

0.1 

0.01-0.32 

23 

0.06 

0.1 

0.01-0.37 

0.53 

KPI:  Delirium  Symptoms 

23 

0.07 

0.04 

0.02-0.16 

23 

0.06 

0.06 

0.01-0.26 

0.89 

KPI:  Antipsychotic  Med  Admin# 

23 

0.04 

0.2 

0.02  -  0.1 

23 

0.06 

0.02 

0.03  -0.12 

0.003 

50 


Award  #:  W81XWH-1 3-1 -0034 
Aurora  Health  Care  -  Year  3  Report 


APPENDIX  G:  KBN  Impact  Study  -  Projected  Timeline  Year  3  with  No  Cost  Extension  thru 
10/2016 


PROJECT  GOALS 

1  -  Identify  essential  knowledge 
and  nursing  practice  behaviors 

2  -  Validate  that  KBN  electronic 

tools  function  as  designed 


3  -  Develop  measures  &  procedure 

for  evaluating  implementation 
and  adoption  of  KBN  practices 

4  -  Conduct  baseline 

measurement  to  identify  gaps 
to  improve  KBN  intervention 
study  and  analyze  findings. 

5  -  Design  the  intervention  study 

strategy  and  delivery  method. 

6  -  Deliver  the  intervention  study 

7  -  Complete  tracking  progress  on 

the  intervention. 

8  -  Complete  full  evaluation 

Conf  with  Military  Advisory  Comm 


l~l  Quartery  &  Annual  Tech.  Reports  Due  ■  I"! 
“  final  reports  due:  4.27.16  S  1.27.17  ™  " 

J Wt  In  Person  Reivew  12.1.15 
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